Apparatus and method for single-step search of neural network architecture using unlabeled data

Abstract

A computer-implemented method for searching for a neural network architecture, where the method comprises training a hyper-model on tagged training data by examining the architecture and utilizing the examined architecture, and training (S21) the hyper-model (1) by updating weights of the hyper-model with respect to a gradient of the examined model. Determining (S23) a Pareto optimal sub-model (1b) of the hyper-model (1) on the basis of the at least two performance metrics by iteratively performing the following steps: calculating an output of the untagged data by a reference model, where the reference model is a largest sub-model (1a) of the hyper-model (1). A plurality of sub-models (1b) are examined from the hyper-model (1). The sub-model (1b) calculates its output to the unlabeled data. A difference between the outputs of the reference model and the sub-model (1b) is calculated. Using an optimization algorithm, the sub-models are iteratively examined and evaluated based on the plurality of objectives.

Description

Technical Field

[0001] This invention relates to a method for searching neural network architectures in a single-shot manner, the method comprising a two-stage approach, a computer program, a machine-readable storage medium, and a system. Background Technology

[0002] The technique of searching neural network architectures (NAS) faces a problem: compared to traditional supervised learning, available data is used not only for 1) training model weights (training data) and 2) validating the model to evaluate its performance on unknown data (validation data), but also for 3) determining the optimal model architecture (search data). Validation data cannot be used for search because it would prevent an unbiased estimation of model performance. Therefore, existing methods also use training data for search.

[0003] Ideally, training and search data should be independent of each other to enable unbiased evaluation of the architecture (and to achieve robustness against potential overfitting to the training data). Techniques such as SPOS (Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, Jian Sun. “SinglePathOne-Shot Neural Architecture Search with Uniform Sampling”. EECV. 2020. https: / / doi.org / 10.1007 / 978-3-030-58517-4_32) follow this approach and split the original training data into two distinct parts for training and search, respectively.

[0004] The problem with this approach is that less data is available for model training, which reduces the quality of the trained model and thus increases the variance during performance evaluation. Therefore, this type of method is only suitable for scenarios where sufficient training data is available (e.g., >50k labeled samples).

[0005] Other techniques, such as BigNAS (Jiahui Yu, Pengchong Jin, Hanxiao Liu, GabrielBender, Pieter-Jan Kindermans, Mingxing Tan, Thomas Huang, Xiaodan Song, Ruoming Pang, Quoc Le. “BigNAS: Scaling up Neural Architecture Search with BigSingle-Stage Models”. ECCV. 2020. https: / / doi.org / 10.1007 / 978-3-030-58571-6_41) or NASViT (Chengyue Gong, DilinWang, Meng Li, Xinlei Chen, Zhicheng Yan, Yuandong Tian, ​​Qiang Liu, Vikas Chandra. “NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware SupernetTraining”), use all the original training data for model training and use some or all of the training data for architecture search, meaning there is overlap between the training data and the search data. While this improves training quality, the search data has already been used during training, making it impossible to provide unbiased estimates of performance on unknown data. This means that such methods are only suitable for scenarios where sufficient training data is available to reduce the risk of overfitting.

[0006] Finally, there have been attempts to perform NAS without labeled data, such as Chenxi Liu, Piotr Dollár, Kaiming He, Ross Girshick, Alan Yuille, and Saining Xie. “AreLabelsNecessary for Neural Architecture Search” ECCV. 2020. http: / / dx.doi.org / 10.1007 / 978-3-030-58548-8_46. These techniques perform NAS on unlabeled proxy tasks (e.g., rotation prediction or color adjustment) and transfer the found architectures to the actual task. While such methods do not require labeled data for NAS, the quality of the found architectures may be poor because the correlation between the performance of the architectures on the proxy task and the actual task may be low. Summary of the Invention

[0007] This invention aims to address the problem of performing single-step NAS when labeled data is scarce. The invention retains all labeled training data for training to maximize the quality of the trained single-step model (supermodel) and uses unlabeled data for architecture search. Specifically, it suggests evaluating the performance of a sub-network by comparing its output for unlabeled data with the output of a reference network. The unlabeled data can come from the same application, from similar data from related applications, or from synthetic data. The choice of the reference network is important because its performance should be higher than that of the sub-networks to produce high-quality reference outputs. Furthermore, this creation should not impose additional training overhead on the already computationally intensive NAS procedure. This invention addresses these challenges by using the largest sub-network in the single-step model as the reference network because 1) this largest sub-network typically has the highest performance among all sub-networks (especially when using specific training techniques such as sandwich rules and in-situ distillation), and 2) this largest sub-network is already available.

[0008] In summary, one objective of this invention is to provide a single-step NAS when there is little available labeled data, wherein this invention effectively addresses this problem by estimating the performance of a subnetwork of the single-step model in the single-step NAS by comparing the output of the subnetwork for unlabeled search data with the output of the largest submodel in the single-step model.

[0009] Advantages of the present invention The aforementioned prior art cannot effectively utilize limited available training data for one-step NAS. In contrast, this invention utilizes all labeled training data for model training, thereby improving the quality of the trained supermodel. This reduces the variance in estimating submodel performance, leading to the discovery of a better architecture. Furthermore, this invention maintains the independence between the training and search data, enabling unbiased estimation of submodel performance, also resulting in a better architecture. Moreover, by avoiding NAS using proxy tasks and instead performing the search on the real-world task, the problem caused by low correlation between proxy and real-world tasks is mitigated, leading to a better architecture.

[0010] Furthermore, since an existing sub-model (e.g., the largest sub-model) is used as a reference model, this invention does not require training and maintaining an additional reference model. This further reduces computational complexity and development effort.

[0011] The disclosure of this invention First, a method for searching neural network architectures is proposed, particularly for training classifiers to classify input signals obtained from sensors. This method begins by training a supermodel with multiple searchable dimensions by examining the architecture within those dimensions, training the supermodel on labeled training data using the examined architecture, and updating the supermodel's weights via gradients relative to the examined model. The supermodel can be a large neural network. Alternatively, it can be a large neural network whose sub-models constitute an efficient architecture with reduced complexity. Examples include fewer channels, fewer layers, and smaller embedding sizes.

[0012] After training, an iterative step is performed: determine the Pareto optimal submodel of the supermodel based on at least two performance metrics.

[0013] Determining the Pareto optimal submodel begins by computing the output of the reference model for unlabeled data, where the reference model is the largest submodel in the supermodel. Performance metrics can be the accuracy of the corresponding submodel and the hardware latency of the corresponding submodel on a specific computing unit or device. Other hardware-related performance metrics can include latency, FLOPS, power consumption, and / or memory footprint. Preferably, performance metrics for the reference model are not computed; only the model outputs (e.g., categories, object bounding boxes, etc.) are computed, and then these model outputs are compared only with each other.

[0014] Next, multiple sub-models within the supermodel are examined. These sub-models then compute their outputs on unlabeled data, a process known as forward propagation of the unlabeled data through the sub-models. The difference between the outputs of the reference model and the sub-models is then determined. Finally, an optimization algorithm is applied to iteratively examine and evaluate the other sub-models based on multiple objectives, including the difference between the outputs of the reference model and the sub-models, as well as other performance metrics. Examples of other performance metrics include latency, FLOPS, power consumption, and / or memory footprint. By iteratively applying the optimization algorithm, the examined sub-models converge toward a Pareto optimal sub-model that achieves the stated objectives.

[0015] In other aspects of the invention, a classifier is proposed to train a supermodel and thus submodels to classify input signals. This classifier can be used by receiving sensor signals including data from sensors, determining input signals dependent on the sensor signals, and feeding the input signals into the classifier to obtain an output signal characterizing the classification of the input signals.

[0016] Such classifiers can then be used to provide actuator control signals to control the actuator, which includes all the steps of the methods described above, and further includes the step of determining the actuator control signal based on the output signal. Preferably, the actuator controls at least partially autonomous robots and / or manufacturing machines and / or access control systems.

[0017] Sensors can determine environmental measurements in the form of sensor signals, which can be provided, for example, by digital images (e.g., video), radar, lidar (LiDAR), ultrasound, motion, thermal imaging, or audio signals.

[0018] Preferably, the present invention can be used to classify sensor data, identify the presence of objects in sensor data, or perform semantic segmentation on sensor data, for example, regarding traffic signs, road surfaces, pedestrians, and vehicles.

[0019] Hypermodels and submodels can be used to determine one or more continuous values, i.e., regression analysis can be performed, for example, regarding distance, velocity, acceleration, or elements in tracking data (e.g., objects). This is performed based on relatively simple features (e.g., edge or pixel attributes of an image).

[0020] This invention can be used to compute control signals to control technical systems, such as computer-controlled machines, like robotic systems, vehicles, home appliances, power tools, manufacturing machines, personal assistants, or access control systems. Attached Figure Description

[0021] The embodiments of the present invention will be explained in more detail with reference to the following figures.

[0022] Figure 1 A schematic architecture of the supermodel and the examined submodels from the supermodel is shown; Figure 2 A schematic flowchart of one embodiment of the present invention is shown; Figure 3 A control system for controlling at least partially autonomous robots is shown; Figure 4 The control system for the manufacturing machine is shown; Figure 5 The control system for controlling the access control system is shown; Figure 6 The control system of the monitoring system is shown; Figure 7 A control system for an automated personal assistant is shown; Figure 8 The control system for the imaging system is shown; Figure 9 A training system for controlling the classifier is shown. Detailed Implementation

[0023] This invention improves the search procedure for two-stage neural network architecture search. Typically, two-stage NAS includes the following two stages: 1) Training the supermodel. This is a neural network model where multiple dimensions are searchable, meaning they are not fixed during supermodel training. Typical searchable dimensions could be the number of layers in different parts of the network, the number of channels, the kernel size, the group size in grouped convolutions, the embedding dimension, the number of attention heads, the MLP ratio, etc.

[0024] Hypermodels enable the extraction of submodels that make specific architectural decisions for searchable dimensions. The maximal submodel uses the maximum value for each searchable dimension. See also Figure 1The diagram illustrates a supermodel 1 and a maximum submodel 1a. The training procedure typically involves examining one or more architectures in the single-step model at each update step, calculating the gradients relative to the selected model, and using this data to update the weights of the supermodel. Known methods in the prior art, such as examining a single random model, are employed. However, the inventors believe that the present invention is particularly well integrated with the sandwich rule of Yu, J., & Huang, TS (“Universally slimmable networks and improved training techniques”. ICCV.2019. https: / / doi.org / 10.48550 / arXiv.1903.05134), which examines two random architectures, as well as the minimum and maximum architectures, at each update iteration, and thus generally results in the maximum model being the best model in the search space. Furthermore, techniques such as “in-situ distillation” (known from earlier publications) facilitate the similarity between the smaller submodel and the maximum submodel.

[0025] After the first phase is completed, the second phase begins: 2) Search Architecture. At this stage, the following sub-models of the supermodel are identified that are Pareto optimal in terms of the objective (e.g., accuracy and / or hardware latency and / or other objectives). This is typically accomplished by iteratively selecting and evaluating the sub-models. Importantly, this step does not require model training because the trainable weights of the sub-models are taken over from the corresponding portions of the trainable weights of the supermodel.

[0026] Optimization algorithms such as evolutionary search are used to improve search efficiency. Evaluation of sub-models includes, for example, measuring accuracy (using search data), latency, etc.

[0027] The iterative process of selecting and evaluating sub-models to find a "good" (i.e., Pareto optimal) sub-model can be performed through evolutionary optimization, in which new sub-models are created by randomly mutating the evaluated sub-models (e.g., changing the number of filters or the depth in a specific part of the network). The idea is that evolutionary optimization continues the search around the best sub-model so far. Another possibility is to use black-box optimization methods, such as Bayesian optimization (BO).

[0028] The main advantage of this invention lies in using unlabeled data for the architecture search phase of a two-stage NAS. This allows more data to be retained for the first stage (i.e., training the supermodel) while maintaining the independence between the training and search data. This is achieved through the following steps: approximating the application performance (e.g., accuracy, IoU) of the submodel by calculating the difference between the submodel's output and the output of the maximum submodel. Figure 1As shown in Appendix 1a. The maximum submodel can be given by all the largest searchable dimensions.

[0029] Given a trained supermodel M and unlabeled search data D, the output pseudocode is as follows: 1) (Optional) Filter search data 2) Calculate the largest submodel A big Output: 3) For each sub-model A : a) Calculate the output of sub-model A: b) Calculate the output difference: Finally, the difference L can be used as a search objective (e.g., minimizing L).

[0030] Unlabeled search data D can be provided from a variety of potential sources. Unlabeled data can be collected from the same application. This may be a case where only a small fraction of the available data is labeled. Additionally or alternatively, unlabeled data can be collected from related applications, for example, data from earlier product generations, which may have been recorded using different camera models. Additionally or alternatively, synthetic data generated from the training data distribution can be used as unlabeled data.

[0031] The selection of unlabeled data can be critical, as a trade-off must be struck between the amount of available data and its similarity to the real-world application. The amount of data should be sufficient to obtain an evaluation of the sub-model's performance with low variance. Preferably, a low-variance evaluation requires, for example, at least 100 input templates, with 1,000 to 10,000 being ideal. Meanwhile, if the unlabeled data is too dissimilar to the application, it may fail to accurately represent the true performance of the sub-model.

[0032] If the range of unlabeled data is much wider than the range of training data—for example, the training data consists of street scenes while the unlabeled data consists of images from the internet—then the output y of the maximum submodel 1a will be greater. big It might not work. In this case, filtering is necessary. D And corresponding filtering y big However, if the range of the training data is similar to the range of the unlabeled search data, filtering is not necessary. Therefore, there are multiple possibilities to utilize f. filt To filter the search data. A simple filtering option is to leave it unfiltered: D'=D. For image classification, filtering can be defined by selecting images with high confidence: D' = .

[0033] For step 3b above, the difference can be calculated as follows: the first option for this difference is the Kullback-Leibler divergence: The second option is mean squared error. , The third option is a hard label: .

[0034] Figure 2 A flowchart 20 is shown for one embodiment of the present invention. Figure 2 The method described is a sophisticated neural network architecture search (NAS) method, a key aspect of machine learning, which aims to automate the design of artificial neural network architectures. Figure 2 The method described here aims to optimize the process of identifying the most efficient and effective neural network architectures from a vast search space that is effectively defined within the hypermodel (1). The method is divided into several core steps, each of which contributes to the overall goal of finding Pareto optimal submodels that strike a balance between performance metrics such as accuracy and hardware metrics such as latency.

[0035] This method begins with training supermodel 1 (S21). This supermodel serves as the foundation from which specific architectures (sub-models) can be examined and evaluated. Training (S21) can comprise several steps. Training begins with architecture examination. Each architecture is examined within the defined searchable dimensions of the supermodel. These dimensions can include aspects such as the number of layers, layer types (convolutional layers, recursive layers, etc.), and layer sizes. The second step is to perform training using labeled training data. Each examined architecture is trained on a dataset with labeled training data. After training the examined architectures, the supermodel's weights are updated. The supermodel's weights are updated based on gradients calculated relative to the performance of the examined architectures. This step ensures that the supermodel learns from the performance of each architecture and progressively improves its ability to generate effective sub-models.

[0036] After training (S21), an unlabeled dataset is provided (S22). This unlabeled dataset can be smaller than the labeled training dataset from step S21. This unlabeled data is crucial for evaluating the performance of the sub-model in a way that does not rely on expensive labels and provides insights into the performance of the sub-model.

[0037] Subsequently, Pareto optimal sub-models (S23) are identified, which provide the best trade-off between at least two performance metrics (such as accuracy and hardware latency), thereby making them Pareto optimal decisions for a particular application.

[0038] Step S23 begins with selecting a reference model. Preferably, the reference model is the largest sub-model (1a). While any other network already trained for the same task can be used as a reference model, this would negate one of the main advantages of the invention—creating a reference model requires no additional cost. The output for the unlabeled data is computed based on the reference model. This reference model should provide a benchmark for comparison. Then, multiple sub-models (1b) are examined from the supermodel (1). Preferably, different sub-models (1b) are randomly examined from the supermodel, where each sub-model represents a different architecture within the search space. The outputs of these sub-models are then computed. Each examined sub-model computes an output for the same unlabeled data. The difference L = f between the output of the reference model and the output of the sub-models is then computed. sim (y, y' big These differences characterize how accurately each sub-model approximates the behavior of the reference model. Based on these differences, an optimization algorithm is used to iteratively examine and evaluate the sub-models. Preferably, the optimization algorithm is applied to multiple objectives, including the calculated differences and their performance metrics. The algorithm attempts to find the sub-models that achieve the best balance between these objectives. After the optimization algorithm converges, the sub-models determined to be Pareto optimal based on these objectives are selected and output, with particular attention to the balance between accuracy and hardware latency. These sub-models represent the most efficient and effective architectures identified during the search process.

[0039] After step S23 is completed, (S24) select a sub-model from the Pareto optimal sub-models to provide on the target device, for example based on a standard predefined by the user for a specific application that should be executed on the target device.

[0040] The sub-model selected according to step S24 can be an image classifier (60), and is preferably used for various applications as described below.

[0041] exist Figure 3 An implementation of an actuator with a control system 40 is shown. The actuator and its environment are collectively referred to as the actuator system. Sensors 30 detect the state of the actuator system at preferably uniformly defined intervals. Sensors 30 may include multiple sensors. Preferably, sensors 30 are optical sensors that record images of the environment. The output signal S of sensors 30 (or, if sensors 30 include multiple sensors, the output signal S of each sensor) encodes the detected state and is transmitted to the control system 40.

[0042] Here, the control system 40 receives a stream of sensor signals S. The control system then calculates a series of actuator control commands A based on the stream of sensor signals S. These actuator control commands are subsequently transmitted to the actuator unit 10, which converts the control commands A into mechanical motion or changes in physical quantities. For example, the actuator unit 10 can convert the control commands A into electrical, hydraulic, pneumatic, thermal, magnetic, and / or mechanical motion or changes. Specific, but not limited, examples include electric motors, electroactive polymers, hydraulic cylinders, piezoelectric actuators, pneumatic actuators, servo mechanisms, electromagnets, stepper motors, etc.

[0043] The control system 40 receives a stream of sensor signals S from the sensor 30 in an optional receiving unit. The receiving unit converts the sensor signals S into input signals x. If there is no receiving unit, each sensor signal S can also be used directly as the input signal x. The input signal x can be given, for example, as an extract of the sensor signals S. Alternatively, the sensor signals S can be processed to generate the input signal x. The input signal x includes image data corresponding to the image recorded by the sensor 30. In other words, the input signal x is provided based on the sensor signals S.

[0044] The input signal x is then passed to classifier 60, which can be provided, for example, by an artificial neural network.

[0045] The classifier 60 consists of parameters stored in and provided by the parameter memory St1. Perform parameterization.

[0046] Classifier 60 determines an output signal y from the input signal x. The output signal y includes information assigning one or more tags to the input signal x. The output signal y is transmitted to an optional converter unit, which converts the output signal y into a control command A. The actuator control command A is then transmitted to actuator unit 10 to correspondingly control actuator unit 10. Alternatively, the output signal y can also be used directly as the control command A.

[0047] Actuator unit 10 receives actuator control command A, is correspondingly controlled, and performs the action corresponding to actuator control command A. Actuator unit 10 may include control logic that converts actuator control command A into another control command, which is then used to control actuator 10.

[0048] In other embodiments, the control system 40 may include the sensor 30. In a further embodiment, the control system 40 may alternatively or additionally include the actuator 10.

[0049] Furthermore, the control system 40 may include a processor 45 (or multiple processors) and at least one machine-readable storage medium 46 on which instructions are stored, which, when executed, cause the control system 40 to perform a method according to one aspect of the invention.

[0050] exist Figure 3 In a preferred embodiment, the control system 40 is used to control an actuator, which is at least partially autonomous robot, such as at least partially autonomous vehicle 100.

[0051] Sensor 30 may include one or more video sensors and / or one or more radar sensors and / or one or more ultrasonic sensors and / or one or more lidar (LiDAR) sensors and / or one or more location sensors (e.g., GPS). Some or all of these sensors are preferably (but not necessarily) integrated into vehicle 100.

[0052] Alternatively or additionally, sensor 30 may include an information system for determining the state of the actuator system. An example of such an information system is a weather information system that determines the current or future weather conditions in the environment.

[0053] For example, classifier 60 can identify objects near, for example, a partially autonomous robot, using an input signal x. The output signal y can include information characterizing where the object is located near the partially autonomous robot. This information can then be used to determine a control command A, for example, to avoid collisions with the identified objects.

[0054] The actuator unit 10, preferably integrated in the vehicle 100, can be provided by the vehicle 100's brakes, drive system, engine, transmission system, or steering system. An actuator control command A can be determined to control the actuator unit (actuator) 10 so that the vehicle 100 avoids collisions with identified objects. The identified objects can also be classified according to what the classifier 60 is most likely to classify them as, for example, pedestrians or trees, and the actuator control command A can be determined based on this classification.

[0055] In other embodiments, the at least partially autonomous robot may be provided by another mobile robot (not shown), which may, for example, fly, swim, dive, or walk. This mobile robot may be primarily a lawnmower or a cleaning robot that is at least partially autonomous. In all the above embodiments, actuator control command A may be determined to control the mobile robot's drive unit and / or steering system and / or brakes, enabling the mobile robot to avoid collisions with identified objects.

[0056] In another embodiment, at least partially autonomous robotic operation may be provided by a gardening robot (not shown) that uses sensor 30, preferably an optical sensor, to determine the state of the plants in environment 20. Actuator unit 10 may be a nozzle for spraying chemicals. Based on the identified species and / or identified state of the plants, actuator control command A can be determined to cause actuator unit 10 to spray the plants with an appropriate amount of suitable chemical.

[0057] In a further embodiment, at least partially autonomous robots can be provided by household appliances (not shown), such as washing machines, stoves, ovens, microwave ovens, or dishwashers. Sensors 30, such as optical sensors, can identify the state of objects that should be handled by the household appliance. For example, if the household appliance is a washing machine, sensor 30 can detect the state of the clothes in the washing machine. The actuator control signal A can then be determined based on the identified clothing material.

[0058] exist Figure 4 One embodiment is shown, in which the control system 40 controls the manufacturing machine 11 of the manufacturing system 200 (e.g., a welding machine, punch press, cutting equipment, or drill gun), for example, as part of a production line. The control system 40 controls the actuator unit 10, which in turn controls the manufacturing machine 11.

[0059] Sensor 30 may be an optical sensor that detects, for example, characteristics of the manufactured product 12. Classifier 60 can determine the state of the manufactured product 12 from these detected characteristics. The actuator unit 10, which controls the manufacturing machine 11, can then be controlled for subsequent manufacturing steps of the manufactured product 12 based on the determined state of the manufactured product 12. Alternatively, it is conceivable that the actuator unit 10 is controlled based on the determined state of the manufactured product 12 during the manufacturing of subsequent products 12.

[0060] exist Figure 5 One embodiment is shown, in which the control system controls the access control system 300. This access control system can be designed for physical access control. For example, the access control system may include a door 401. Sensor 30 is designed to identify scenarios relevant to deciding whether to grant access. For example, the access control system may be an optical sensor that provides image or video data to identify faces. Classifier 60 can be designed to interpret this image or video data in such a way that the classifier, for example, compares it to a known person stored in a database to determine the person's identity. An actuator control signal A can then be determined based on the interpretation of classifier 60 (e.g., based on the determined identity). Actuator unit 10 may be a lock that allows or prohibits access based on actuator control signal A. Non-physical, logical access control is also possible.

[0061] exist Figure 6 One embodiment is shown, in which the control system 40 controls the monitoring system 400. This embodiment is similar to... Figure 5 The embodiments shown are largely the same. Therefore, only the differences will be described in detail here. Sensor 30 is designed to identify the monitored scene. The control system does not necessarily control actuator 10, but rather controls display 10a. Thus, machine learning system 60 can determine the classification of the scene, for example, whether the scene detected by optical sensor 30 is suspicious. Actuator control signal A transmitted to display 10a can then be designed, for example, to adapt the displayed content according to the determined classification, for example, to highlight objects classified as suspicious by machine learning system 60.

[0062] exist Figure 7 One embodiment is shown, in which control system 40 controls automated personal assistant 250. Sensor 30 may be an optical sensor, which, for example, receives a video image of a gesture from user 249. Alternatively, sensor 30 may also be an acoustic sensor, for example, for receiving voice commands from user 249.

[0063] The control system 40 then determines the actuator control command A for controlling the automated personal assistant 250. The actuator control command A is determined based on the sensor signal S from the sensor 30. The sensor signal S is sent to the control system 40. For example, the classifier 60 can be designed to execute, for instance, a gesture recognition algorithm to identify the gestures of the user 249. The control system 40 can then determine the actuator control command A to transmit to the automated personal assistant 250. The control system 40 then transmits this actuator control command A to the automated personal assistant 250.

[0064] Therefore, for example, the actuator control command A can be determined based on the user's gestures identified by the classifier 60. This actuator control command can then include information that enables the automated personal assistant 250 to retrieve information from a database and output this retrieved information in a format suitable for the user 249 to receive.

[0065] In other embodiments, it is conceivable that the control system 40 does not control the automated personal assistant 250, but rather controls a household appliance (not shown) that is controlled based on recognized user gestures. This appliance could be a washing machine, stove, oven, microwave oven, or dishwasher.

[0066] exist Figure 8The diagram illustrates an implementation of a control system 40 for controlling the imaging system 500, such as an MRT device, X-ray imaging device, or ultrasound imaging device. The sensor 30 may be, for example, an imaging sensor. The machine learning system 60 can then classify all or part of the detected image. An actuator control signal A can then be selected based on this classification, thereby controlling the display 10a. For example, the machine learning system 60 may interpret a region of the detected image as a potential anomaly. In this case, the actuator control signal A can be determined to cause the display 10a to display the image and highlight the potentially anomalous region.

[0067] exist Figure 9 The diagram illustrates an implementation of the training system 500. The training system 500 comprises a providing system 51 that provides input images from a training dataset. The input images are fed to a neural network 52 (e.g., a hypermodel) to be trained, from which the neural network determines output variables. The output variables and the input images are fed to an evaluator 53, from which the evaluator determines temporary hyperparameters, which are then transferred to a parameter memory P, where they replace the current parameters. The evaluator 53 can be configured to perform operations based on… Figure 1 Step S21 of the method.

[0068] The method executed by the training device 500 can be implemented as a computer program, which is stored on a machine-readable storage medium 54 and executed by a processor 55.

[0069] The term "computer" includes any device used to process predefined computational instructions. These computational instructions can exist in software, hardware, or a combination of both.

[0070] It should also be understood that these methods can be implemented not only entirely in the described software, but also in hardware or in a hybrid of software and hardware.

Claims

1. A method (20) for a computer implementation of a search neural network architecture, wherein the method comprises: The supermodel (1) with multiple searchable dimensions is trained (S21) by: examining the architecture within the searchable dimensions and training the examined architecture on labeled training data, and updating the weights of the supermodel with training gradients from the examined architecture. By iteratively performing the following steps, the Pareto optimal submodel (1b) of the hypermodel (1) is determined based on at least two performance metrics: (S23) Calculate the output of the reference model for unlabeled data, wherein the reference model is the largest submodel (1a) of the hypermodel (1). Examine multiple sub-models (1b) from the supermodel (1); The sub-model (1b) calculates its output for the unlabeled data; Calculate the difference between the outputs of the reference model and the sub-model (1b); An optimization algorithm is used to iteratively examine and evaluate the sub-model based on multiple objectives, including the difference, the examined architecture of the sub-model, and performance metrics, particularly accuracy and hardware latency; and Based on the stated objective, and preferably based on the stated performance metric, the Pareto optimal sub-model is output.

2. The method of claim 1, wherein, in order to train the supermodel, at least two architectures are examined in each training step, including a minimum architecture and a maximum architecture based on searchable dimensions.

3. The method according to claim 1 or 2, wherein the unlabeled data is data obtained from the same or related application, or synthetic data generated from the data distribution of the labeled training data.

4. The method according to any one of claims 1 to 3, wherein the optimization algorithm is evolutionary optimization or Bayesian optimization.

5. The method according to any one of claims 1 to 4, wherein the difference is Kullback-Leibler divergence, mean square error, or hard label difference.

6. The method according to any one of claims 1 to 5, wherein the unlabeled data has been filtered, wherein the filtering is performed by selecting data points of the unlabeled dataset, wherein the confidence of the supermodel (1) is higher than a predefined threshold.

7. The method according to any one of claims 1 to 6, wherein the supermodel and the submodel are trained as a classifier (60) for classifying sensor signals, wherein after training, the classifier (60) is designed to receive a sensor signal (S) containing data from a sensor (30), and to determine an input signal (x) dependent on the sensor signal (S), in particular an image, and to feed the input signal (x) to the classifier (60) to obtain an output signal (y) characterizing the classification of the input signal (x).

8. The method according to any one of claims 1 to 7, wherein a sub-model is selected from the Pareto optimal sub-model based on a predefined criterion, and the selected sub-model is used to provide an actuator control signal (A) for controlling the actuator (10), in particular by determining the actuator control signal (A) according to the output of the sub-model, in particular according to the output signal (y).

9. The method of claim 8, wherein the actuator (10) controls at least partially autonomous robot (100) or vehicle or manufacturing machine (200) or access control system (300).

10. A computer program designed to cause a computer to perform all the steps of the method according to any one of claims 1 to 9 when the computer program is executed by a processor (45, 145).

11. A machine-readable storage medium (46, 146) having a computer program as claimed in claim 10 stored thereon.

12. A system designed to perform the method according to any one of claims 1 to 9.

Images