Device and method for searching for neural architecture in a single step using unlabeled data
The two-stage NAS method addresses limited labeled data challenges by using unlabeled data for architecture search, comparing subnetworks to a reference network, enhancing model quality and reducing computational costs.
Patent Information
- Application Number
- DE102024207608
- Authority / Receiving Office
- DE · DE
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2026-02-12
AI Technical Summary
Existing neural architecture search (NAS) methods face challenges with limited labeled data, leading to reduced model quality, variance in performance estimates, and biased assessments due to data overlap between training and search phases, or reliance on low-correlation proxy tasks.
A two-stage NAS method that reserves labeled data for training and uses unlabeled data for architecture search, evaluating subnetworks by comparing their outputs to a reference network, particularly the largest subnetwork, without additional training costs.
Improves model quality by maintaining data independence, reducing variance, and avoiding proxy task issues, resulting in better architecture discovery with reduced computational complexity.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[0001] The invention relates to a method for searching for neural architecture in a single step (“one-shot”), comprising a two-stage method, a computer program and a machine-readable storage medium as well as a system. State of the art
[0002] Neural Architecture Search (NAS) techniques face the challenge that, compared to classical supervised learning, the available data must be used not only for 1) training the model weights (training data) and 2) validating the model to estimate its performance on unknown data (validation data), but also 3) for determining the optimal model architecture (search data). The validation data must not be used for the search itself, as this would compromise the unbiased assessment of model performance. Therefore, existing methods also utilize the training data for the search.
[0003] Ideally, training and search data are independent to allow for an unbiased evaluation of architectures (and to achieve robustness against potential overfitting to the training data). Techniques such as SPOS (Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, Jian Sun. “Single Path One-Shot Neural Architecture Search with Uniform Sampling”. EECV. 2020. https: / / doi.org / 10.1007 / 978-3-030-58517-4_32) follow this approach and split the original training data into two separate parts for training and searching.
[0004] The problem with such an approach is that less data is available for model training, which reduces the quality of the trained models and thus increases the variance in performance estimates. Therefore, such approaches are only applicable to scenarios where sufficient training data is available (e.g., >50k labeled examples).
[0005] Other techniques, such as BigNAS (Jiahui Yu, Pengchong Jin, Hanxiao Liu, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Thomas Huang, Xiaodan Song, Ruoming Pang, Quoc Le. “BigNAS: Scaling up Neural Architecture Search with Big Single-Stage Models”. ECCV. 2020. https: / / doi.orgJ10.1007 / 978-3-030-58571-6_41) or NASViT (Chengyue Gong, Dilin Wang, Meng Li, Xinlei Chen, Zhicheng Yan, Yuandong Tian, Qiang Liu, Vikas Chandra. “NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet Training”), use the complete original training data for model training and also use parts of or the complete training data for architecture search; that is, there is an overlap between training and Search data. While this improves the quality of the training, the search data was used during the training and therefore does not provide an unbiased assessment of performance with unknown data.This means that such approaches are only applicable in scenarios where sufficient training data is available to reduce the risk of overfitting.
[0006] Finally, there have been attempts to perform NAS without labeled data, e.g., Chenxi Liu, Piotr Dollár, Kaiming He, Ross Girshick, Alan Yuille, Saining Xie. “Are Labels Necessary for Neural Architecture Search?”. ECCV. 2020. http: / / dx.doi.org / 10.1007 / 978-3-030-58548-8_46. These techniques perform the NAS on an unlabeled proxy task, such as rotation prediction or recoloring, and transfer the discovered architecture to the actual task. While such methods do not require labeled data for the NAS, they suffer from lower quality architecture discoveries because the correlation between an architecture's performance on the proxy task and the actual task may be weak.
[0007] The present invention aims to solve the problem of a single-stage NAS with limited available labeled data. The invention reserves all labeled training data for training to maximize the quality of the trained one-shot model (supermodel) and uses unlabeled data for architecture search. Specifically, it proposes evaluating the performance of a particular subnetwork of a one-shot model by comparing its output to unlabeled data with the output of a reference network. The unlabeled data could be from the same application, similar data from a related application, or synthetic data. The selection of the reference network is important because its performance should be higher than that of the subnetwork to produce high-quality reference output.Furthermore, the creation should not entail any additional training costs for the already computationally intensive NAS procedure. The invention addresses these challenges by using the largest subnetwork in the one-shot model as the reference network, since it 1) typically exhibits the highest performance among all subnetworks (especially when certain training techniques such as the sandwich rule and in-place distillation are used) and 2) is already available.
[0008] In summary, one object of the invention is to provide one-shot NAS with little available labeled data, wherein the invention effectively solves this problem by estimating the performance of subnetworks of the one-shot model in the one-shot NAS by comparing their output to unlabeled search data with the output of the largest submodel in the one-shot model. Advantages of the invention
[0009] The prior art described above cannot efficiently utilize the limited available training data for one-shot NAS. In contrast, the present invention uses all labeled training data for model training, thereby improving the quality of the trained supermodel. This reduces the variance in submodel performance estimates, leading to better discovered architectures. Furthermore, the present invention maintains the independence between training and search data, enabling unbiased assessments of submodel performance and also resulting in better discovered architectures. Additionally, by avoiding a proxy task for the NAS and searching on the actual task, problems arising from the low correlation between the proxy task and the actual task are mitigated, leading to better discovered architectures.
[0010] Furthermore, since an existing submodel (e.g., the largest submodel) is used as a reference model, no training or maintenance of an additional reference model is required for this invention. This further reduces computational complexity and development effort. Disclosure of the invention
[0011] First, a method for searching neural architectures is proposed, specifically for training a classifier to classify input signals received from a sensor. The method begins by training a multi-searchable-dimension supermodel by testing an architecture within the searchable dimension. The supermodel is then trained on labeled training data using the tested architecture, and its weights are updated by gradients with respect to the tested models. The supermodel can be a large neural network, with submodels of this network forming valid architectures of reduced complexity. Examples include fewer channels, fewer layers, smaller embedding sizes, and so on.
[0012] After training, an iterative step is performed to determine Pareto-optimal submodels of the supermodel based on at least two performance indicators.
[0013] Determining Pareto-optimal submodels begins by calculating the outputs of a reference model for the unlabeled data, where the reference model is a largest submodel of the supermodel. Performance metrics can include the accuracy of each submodel and its hardware latency on a specific computing unit or device. Other hardware-related performance metrics can include latency, FLOPS, power consumption, and / or memory utilization. Preferably, the performance metrics for the reference model are not calculated; instead, only model outputs (e.g., classes, object bounding boxes, etc.) are computed and then compared.
[0014] Then, a plurality of submodels from the supermodel are tested. The submodels then compute their outputs of the unlabeled data, also known as unlabeled data propagation by the submodels. Next, a difference between the outputs of the reference model and the submodels is determined. Finally, an optimization algorithm is applied to iteratively test and evaluate further submodels based on a plurality of objectives, where the objectives include the difference between the outputs of the reference model and the submodels, as well as other performance metrics. Examples of other performance metrics include latency, FLOPS, power consumption, and / or memory utilization. Through the iterative application of the optimization algorithm, the tested submodels converge toward the Pareto-optimal submodels for the stated objectives.
[0015] Further aspects of the invention provide that the supermodel, and thus the submodels, are trained with a classifier to classify input signals. The classifier can be used by receiving a sensor signal comprising data from a sensor, determining an input signal that depends on the sensor signal, and feeding the input signal into the classifier to obtain an output signal that characterizes a classification of the input signal.
[0016] Such classifiers can then be used to provide an actuator control signal for controlling an actuator, comprising all the steps of the above method and further including the step of determining the actuator control signal as a function of the output signal. Preferably, the actuator controls an at least partially autonomous robot and / or a manufacturing machine and / or an access control system.
[0017] The sensor can determine measurements of the environment in the form of sensor signals, which can be provided, for example, by digital images, e.g., video, radar, LiDAR, ultrasound, motion, thermal images or audio signals.
[0018] Preferably, the invention can be used to classify the sensor data, detect the presence of objects in the sensor data, or perform semantic segmentation of the sensor data, e.g., with regard to traffic signs, road surfaces, pedestrians, and vehicles.
[0019] The supermodel and submodels can be used to determine one or more continuous values; that is, a regression analysis can be performed, for example, regarding distance, velocity, acceleration, or the tracking of an element, such as an object, in the data. This is done based on relatively simple features (e.g., edges or pixel attributes for images).
[0020] The invention can be used to calculate a control signal for controlling a technical system, such as a computer-controlled machine, a robot system, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, or an access control system.
[0021] Embodiments of the invention are explained in more detail with reference to the following figures. They show: Fig. 1 a schematic architecture of a supermodel and verified submodels from the supermodel; Fig. 2 a schematic flowchart of an embodiment of the invention; Fig. 3 a control system that controls an at least partially autonomous robot; Fig. 4 a control system that controls a manufacturing machine; Fig. 5 a control system that controls an access control system; Fig. 6 a control system that controls a monitoring system; Fig. 7 a control system that controls an automated personal assistant; Fig. 8 a control system that controls an imaging system; Fig. 9 a training system for controlling the classifier.
[0022] The present invention improves the search procedure of the two-stage neural architecture search. In general, the two-stage NAS comprises the following two stages: 1) Training the supermodel. This involves a neural network model where several dimensions of the model are searchable, meaning they are not fixed during supermodel training. Typical searchable dimensions might include the number of layers in different parts of the network, the number of channels, kernel sizes, group sizes in clustered convolutions, embedding dimensions, the number of attentional heads, MLP ratios, etc.
[0023] A supermodel allows the extraction of submodels that make specific architectural decisions for the searchable dimensions. The largest submodel uses the maximum values for each searchable dimension. See Fig. Figure 1 represents a supermodel 1 and the largest submodel 1a. The training procedure typically involves testing one or more architectures from the one-shot model in each update iteration, calculating the gradient with respect to the selected models, and using this data to update the weights of the supermodel. Known prior art approaches test, for example, a single random model. However, the inventors believe that the present invention is particularly well-suited to the sandwich rule of Yu, J., & Huang, TS, "Universally slimmable networks and improved training techniques." ICCV. 2019. https: / / doi.org / 10.48550 / arXiv.1903.05134, which tests two random architectures as well as the smallest and largest architectures in each update iteration, thus generally resulting in the largest model being the best model in the search space.Furthermore, techniques such as in-place distillation (known from previous publications) promote the similarity of smaller sub-models to the largest.
[0024] After completion of the first phase, the second phase begins: 2) Architecture search. In this phase, submodels of the supermodel are identified that are Pareto-optimal with respect to the objectives (e.g., accuracy and / or hardware latency and / or other goals). This is usually done through iterative selection of submodels and their evaluation. Importantly, this step does not require model training, as the trainable weights of the submodel are inherited from the corresponding parts of the trainable weights of the supermodel.
[0025] The use of optimization algorithms such as evolutionary search improves search efficiency. The evaluation of submodels includes, for example, measuring accuracy (using search data), latency, etc.
[0026] The process of iteratively selecting submodels for evaluation to find the "good" (i.e., Pareto-optimal) ones can be performed through evolutionary optimization, where new submodels are created by randomly mutations (e.g., changing the number of filters in a particular part of the network or altering the depth) of already evaluated submodels. The idea is that evolutionary optimization continues the search close to the best submodels found so far. Another possibility would be to use black-box optimization techniques such as Bayesian optimization (BO).
[0027] The main advantage of this invention is the use of unlabeled data for the architecture search phase of the two-stage NAS. This leaves more data available for the first stage, i.e., training the supermodel, while maintaining the independence between training and search data. This is achieved by the following steps to approximate the performance (e.g., accuracy, loU) of submodels by calculating the difference between their output and the output of the largest submodel, as shown in Fig. Represented by the reference symbol 1a. The largest submodel can be given by all maximally searchable dimensions.
[0028] Given a trained supermodel M and unlabeled search data D, a pseudocode is output as follows:
[0029] Finally, the difference L can be used as the search goal (e.g., to minimize L).
[0030] The unlabeled search data D can be provided from various potential sources. Unlabeled data can be collected from the same application. This might be the case if only a fraction of the available data has been labeled. Additionally or alternatively, unlabeled data can be collected from a related application, such as data from a previous product generation, perhaps captured with a different camera model. Additionally or alternatively, synthetic data generated from the training data distribution can be used as unlabeled data.
[0031] The selection of unlabeled data can be crucial, as a trade-off must be struck between the amount of available data and its similarity to the actual application. The data set should be sufficient to obtain low-variance estimates of the submodel's performance. Preferably, low-variance estimates require, for example, at least 100 input patterns, with 1,000 to 10,000 being ideal. At the same time, if the unlabeled data is too dissimilar to the application, it may not accurately reflect the submodel's performance.
[0032] If the range of unlabeled data is much wider than that of the training data (for example: training data consists of street scenes, unlabeled data consists of images from the internet), the output y may be big of the largest submodel 1a is not useful. In this case, D and accordingly y should be used. bigThe training data and the unlabeled search data can be filtered. However, if the range of the training data and the unlabeled search data are similar, no filtering is necessary. Therefore, there are several ways to filter search data using f. filt can be filtered. The trivial filter option can also not be a filter: D' = D. For image classification, the filter operation can be defined by selecting images with high confidence: D' = {d|max(M(d,A)} big )) > δ, d ∈ D}.
[0033] For step 3b) above, the difference can be calculated as follows: One option for the difference is a Kullback-Leibler divergence: L=1|Y|∑(ybig,y)∈(Ybig,Y)∑jybigjlog(ybigjyj)), A second option is the mean squared error: L=1|Y|∑(ybig,y)∈(Ybig,Y)∑j(ybigj−yj)2, A third option is hard labels: L=1|Y|∑(ybig,y)∈(Ybig,Y):max(ybig)≠max(y).
[0034] Fig. Figure 2 shows a flowchart 20 of an embodiment of the invention. The in Fig. The method described in section 2 is a sophisticated approach for the search for neural architecture (NAS), a crucial aspect of machine learning that aims to automate the design of artificial neural network architectures. The method in Fig. The goal of Section 2 is to optimize the process of identifying the most efficient and effective neural network architectures from a huge search space effectively defined within a supermodel (1). The procedure is divided into several core steps, each contributing to the overall goal of finding Pareto-optimal submodels that strike a balance between performance metrics such as accuracy and hardware metrics such as latency.
[0035] The procedure begins with training (S21) of Supermodel 1. This supermodel serves as the foundation from which specific architectures (submodels) can be tested and evaluated. The training (S21) can comprise several steps. It begins with an architecture check. Within the defined searchable dimensions of the supermodel, individual architectures are tested. These dimensions could include aspects such as the number of layers, the type of layers (convolutional, recursive, etc.), and the layer sizes. In the second step, training is performed using labeled training data. Each tested architecture is trained using a dataset of labeled training data. After training the tested architectures, a weight update of the supermodel is performed. The weights of the supermodel are updated based on gradients calculated with respect to the performance of the tested architectures.This step ensures that the supermodel learns from the performance of each architecture and gradually improves its ability to generate effective submodels.
[0036] After training (S21), an unlabeled dataset is provided (S22). The unlabeled dataset may be smaller than the labeled training dataset from step S21. This unlabeled data is crucial for evaluating the performance of the submodels in a way that is independent of expensive labels and provides insights into the performance of the submodels.
[0037] Subsequently, the Pareto-optimal submodels are determined (S23), which are submodels that offer the best compromise between at least two performance metrics such as accuracy and hardware latency, thus becoming Pareto-optimal decisions for certain applications.
[0038] Step S23 begins with the selection of a reference model. Preferably, the reference model is the largest submodel (1a). It would be possible to use any other network trained for the same task as the reference model, but this would compromise one of the main advantages of this invention—no additional cost for creating the reference model. The outputs for the unlabeled data are calculated using a reference model. This reference model is intended to provide a benchmark. Then, a plurality of submodels (1b) from the supermodel (1) are examined. Preferably, different submodels (1b) are examined randomly from the supermodel, each representing a different architecture within the search space. Then, the outputs for the submodels are calculated. Each examined submodel computes outputs for the same unlabeled data. Then, a difference is calculated. L=fsim(y,ybig') Differences are calculated between the outputs of the reference model and those of the submodels. These differences characterize how closely each submodel approximates the behavior of the reference model. Based on these differences, an optimization algorithm is used to iteratively examine and evaluate submodels. Preferably, the optimization algorithm is applied with respect to a plurality of objectives, including the calculated differences and their performance metrics. The algorithm attempts to find submodels that achieve the most balanced ratio according to these objectives. After the optimization algorithm converges, the submodels determined to be Pareto-optimal based on the objectives, with particular attention to a balanced ratio of accuracy and hardware latency, are selected and output. These submodels represent the most efficient and effective architectures identified during the search process.
[0039] After completion of step S23, a submodel of the Pareto-optimal submodels (S24) is selected for deployment on a target device, based, for example, on a user-defined criterion of a specific application to be run on the target device.
[0040] The submodel selected according to step S24 can be an image classifier (60) and can preferably be used for various applications as described below.
[0041] In Fig. Figure 3 shows an embodiment of an actuator with a control system 40. The actuator and its environment are collectively referred to as the actuator system. A sensor 30 detects a state of the actuator system at preferably uniformly dimensioned intervals. The sensor 30 can comprise several sensors. Preferably, the sensor 30 is an optical sensor that captures images of the environment. An output signal S of the sensor 30 (or, if the sensor 30 comprises a plurality of sensors, an output signal S for each of the sensors), which encodes the detected state, is transmitted to the control system 40.
[0042] The control system 40 receives a stream of sensor signals S. Depending on the current of the sensor signals S, it then calculates a series of actuator control commands A, which are then transmitted to the actuator unit 10. The actuator unit 10 converts the control commands A into mechanical movements or changes in physical quantities. For example, the actuator unit 10 can convert the control command A into an electrical, hydraulic, pneumatic, thermal, magnetic, and / or mechanical movement or change. Specific, but not limited, examples include electric motors, electroactive polymers, hydraulic cylinders, piezoelectric actuators, pneumatic actuators, servomechanisms, electromagnets, stepper motors, etc.
[0043] The control system 40 receives the current of the sensor signals S from sensor 30 in an optional receiver unit. The receiver unit converts the sensor signals S into input signals x. If no receiver unit is available, any sensor signal S can alternatively be used directly as an input signal x. The input signal x can, for example, be specified as an extract from the sensor signal S. Alternatively, the sensor signal S can be processed to produce an input signal x.
[0044] The input signal x comprises image data corresponding to an image captured by sensor 30. In other words, the input signal x is provided according to the sensor signal S.
[0045] The input signal x is then forwarded to the classifier 60, which can be, for example, an artificial neural network.
[0046] The classifier 60 is parameterized by parameters ϕ, which are stored in and provided by the parameter memory St1.
[0047] The classifier 60 determines output signals y from the input signals x. The output signal y contains information that assigns one or more labels to the input signal x. The output signals y are transmitted to an optional converter unit, which converts the output signals y into control commands A. The actuator control commands A are then transmitted to the actuator unit 10 to control the actuator unit 10 accordingly. Alternatively, the output signals y can also be used directly as control commands A.
[0048] The actuator unit 10 receives actuator control commands A, is controlled accordingly, and executes an action corresponding to the actuator control commands A. The actuator unit 10 may include control logic that converts the actuator control command A into another control command, which is then used to control the actuator 10.
[0049] In further embodiments, the control system 40 can include a sensor 30. In still further embodiments, the control system 40 can alternatively or additionally include an actuator 10.
[0050] Furthermore, the control system 40 can comprise a processor 45 (or a plurality of processors) and at least one machine-readable storage medium 46 on which instructions are stored which, when executed, cause the control system 40 to execute a method according to an aspect of the invention.
[0051] In a preferred embodiment of Fig. 3 The control system 40 is used to control the actuator, which is an at least partially autonomous robot, e.g. an at least partially autonomous vehicle 100.
[0052] The sensor 30 can comprise one or more video sensors and / or one or more radar sensors and / or one or more ultrasonic sensors and / or one or more LiDAR sensors and / or one or more position sensors (such as GPS). Some or all of these sensors are preferably, but not necessarily, integrated into vehicle 100. Alternatively or additionally, the sensor 30 can comprise an information system for determining a state of the actuator system. An example of such an information system is a weather information system that determines a current or future state of the weather in the surrounding area.
[0053] For example, using the input signal x, the classifier 60 can detect objects near the at least partially autonomous robot. The output signal y can contain information characterizing the location of objects near the at least partially autonomous robot. The control command A can then be determined in accordance with this information, e.g., to avoid collisions with the detected objects.
[0054] The actuator unit 10, which is preferably integrated into vehicle 100, can be a brake, a drive system, an engine, a powertrain, or a steering system of vehicle 100. Actuator control commands A can be determined such that the actuator unit (the actuators) 10 is / are controlled in such a way that the vehicle 100 avoids collisions with the detected objects. Detected objects can also be classified according to what the classifier 60 most likely classifies them as, e.g., pedestrians or trees, and actuator control commands A can be determined depending on the classification.
[0055] In further embodiments, the at least partially autonomous robot can be another mobile robot (not shown) that can move, for example, by flying, swimming, diving, or walking. The mobile robot can be, among other things, an at least partially autonomous lawnmower or a at least partially autonomous cleaning robot. In all the embodiments mentioned above, the actuator control command A can be determined such that the drive unit and / or the steering and / or the brake of the mobile robot are controlled in such a way that the mobile robot can avoid collisions with the detected objects.
[0056] In another embodiment, the at least partially autonomous robot can be a garden robot (not shown) that uses a sensor 30, preferably an optical sensor, to determine the condition of plants in the environment 20. The actuator 10 can be a nozzle for spraying chemicals. Depending on the detected type and / or condition of the plants, an actuator control command A can be determined to cause the actuator 10 to spray the plants with a suitable quantity of suitable chemicals.
[0057] In further embodiments, the at least partially autonomous robot can be represented by a household appliance (not shown), such as a washing machine, a stove, an oven, a microwave, or a dishwasher. The sensor 30, e.g., an optical sensor, can detect the state of an object that is to be processed by the household appliance. If the household appliance is, for example, a washing machine, the sensor 30 can detect the state of the laundry inside the washing machine. The actuator control signal A can then be determined depending on the detected material of the laundry.
[0058] In Fig. Figure 4 shows an embodiment in which the control system 40 is used to control a manufacturing machine 11 (e.g. a soldering machine, a punch, a cutting device or a drilling gun) of a manufacturing system 200, e.g. as part of a production line.
[0059] The control system 40 controls an actuator unit 10, which in turn controls the production machine 11.
[0060] Sensor 30 can be an optical sensor that detects the properties of, for example, a manufactured product 12. Classifier 60 can determine a state of the manufactured product 12 from these detected properties. Actuator 10, which controls the production machine 11, can then be controlled for a subsequent production step of the manufactured product 12, depending on the determined state of the manufactured product 12. Alternatively, it is conceivable that actuator 10 is controlled during the production of a subsequent manufactured product 12, depending on the determined state of the manufactured product 12.
[0061] In Fig. Figure 5 shows an embodiment in which the control system controls an access control system 300. The access control system can be designed for physical access control. For example, it can include a door 401. The sensor 30 is designed to detect a scene relevant to the decision of whether or not to grant access. It can, for example, be an optical sensor that provides image or video data to recognize a person's face. The classifier 60 can be designed to interpret this image or video data by, for example, comparing it with known persons stored in a database and thus determining the person's identity. The actuator control signal A can then be determined depending on the interpretation of the classifier 60, for example, according to the determined identity.The actuator unit 10 can be a lock that allows or denies access depending on the actuator control signal A. Non-physical, logical access control is also possible.
[0062] In Fig. Figure 6 shows an embodiment in which the control system 40 controls a monitoring system 400. This embodiment is largely identical to the one in Figure 6. Fig. The embodiment shown in Figure 5 is not included. Therefore, only the differing aspects are described in detail here. The sensor 30 is designed to detect a scene under surveillance. The control system does not necessarily control an actuator 10, but rather a display 10a. For example, the machine learning system 60 can determine a scene classification, such as whether the scene detected by the optical sensor 30 is suspicious. The actuator control signal A, which is transmitted to the display 10a, can then be designed, for example, to cause the display 10a to adjust its displayed content depending on the determined classification, e.g., to highlight an object that the machine learning system 60 classifies as suspicious.
[0063] In Fig. Figure 7 shows an embodiment in which the control system 40 is used to control an automatic personal assistant 250. The sensor 30 can be an optical sensor that, for example, receives video images of gestures made by the user 249. Alternatively, the sensor 30 can also be an acoustic sensor, for example, to receive a voice command from the user 249.
[0064] The control system 40 then determines the actuator control commands A for controlling the automated personal assistant 250. The actuator control commands A are determined according to the sensor signal S of the sensor 30. The sensor signal S is sent to the control system 40. The classifier 60 can, for example, be configured to execute a gesture recognition algorithm to identify a gesture of the user 249. The control system 40 can then determine an actuator control command A for transmission to the automated personal assistant 250. It then transmits this actuator control command A to the automated personal assistant 250.
[0065] For example, the actuator control command A can be determined according to the user gesture recognized by the classifier 60. It can then include information that causes the automated personal assistant 250 to retrieve information from a database and output this retrieved information in a form suitable for receipt by the user 249.
[0066] In further embodiments, it is conceivable that the control system 40, instead of the automated personal assistant 250, controls a household appliance (not shown) which is operated according to the identified user gesture. The household appliance can be a washing machine, a stove, an oven, a microwave, or a dishwasher.
[0067] In Fig. Figure 8 shows an embodiment of a control system 40 for controlling an imaging system 500, for example, an MRI scanner, an X-ray imaging system, or an ultrasound imaging system. The sensor 30 can, for example, be an imaging sensor. The machine learning system 60 can then perform a classification of all or part of the acquired image. The actuator control signal A can then be selected according to this classification, thereby controlling the display 10a. For example, the machine learning system 60 can interpret an area of the acquired image as potentially anomalous. In this case, the actuator control signal A can be determined to cause the display 10a to show the imaging and highlight the potentially anomalous area.
[0068] In Fig. Figure 9 shows an embodiment of a training system 500. The training device 500 consists of a provisioning system 51 that provides input images from a training dataset. The input images are fed to the neural network 52 to be trained (e.g., a supermodel), which determines output variables from them. Output variables and input images are fed to an evaluator 53, which determines acute hyperparameters from them. These hyperparameters are transferred to the parameter memory P, where they replace the current parameters. The evaluator 53 can be configured to perform steps S21 of the procedure according to Fig. 1 to execute.
[0069] The procedures performed by the training device 500 can be implemented as a computer program stored on a machine-readable storage medium 54 and executed by a processor 55.
[0070] The term "computer" encompasses any device for processing predefined computational instructions. These instructions can be in the form of software, hardware, or a hybrid of both.
[0071] It is further understood that the methods can be implemented not only entirely in the described software, but also in hardware or in a mixed form of software and hardware. QUOTES INCLUDED IN THE DESCRIPTION
[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited non-patent literature
[0000] Jiahui Yu, Pengchong Jin, Hanxiao Liu, Gabriel Bender, Pieter-Jan Kindermans, Mingxing Tan, Thomas Huang, Xiaodan Song, Ruoming Pang, Quoc Le. „BigNAS: Scaling up Neural Architecture Search with Big Single-Stage Models“. ECCV. 2020. https: / / doi.orgJ10.1007 / 978-3-030-58571-6_41
[0005] Chenxi Liu, Piotr Dollár, Kaiming He, Ross Girshick, Alan Yuille, Saining Xie. „Are Labels Necessary for Neural Architecture Search?“. ECCV. 2020. http: / / dx.doi.org / 10.1007 / 978-3-030-58548-8_46.
[0006] Yu, J., & Huang, T. S. „Universally slimmable networks and improved training techniques“. ICCV. 2019. https: / / doi.org / 10.48550 / arXiv.1903.05134
[0023]
Claims
[1] Computer-implemented method (20) for searching for neural architecture, wherein the method comprises: Training (S21) a supermodel (1) with multiple searchable dimensions by testing architectures within the searchable dimension and training the tested architectures on labeled training data and updating the weights of the supermodel by gradients of training from the tested architectures; Determining (S23) Pareto-optimal submodels (1b) of the supermodel (1) based on at least two performance indicators by iteratively performing the following steps: Computing the outputs of a reference model for unlabeled data, where the reference model is a largest submodel (1a) of the supermodel (1); Checking a plurality of submodels (1b) from the supermodel (1); Calculate, through the submodels (1b), their outputs of the unlabeled data; Calculating a difference between the outputs of the reference model and the submodels (1b); Using an optimization algorithm for iterative testing and Evaluating sub-models based on a plurality of objectives, the objectives include the differences and the tested architectures of the sub-models and performance indicators, in particular accuracy and hardware latency; and Output of the Pareto-optimal submodels based on the stated objectives, preferably based on the performance indicators. [2] Method according to claim 1, wherein for training the supermodel at least two architectures are tested for each training step, including a smallest and a largest architecture based on the searchable dimensions. [3] Method according to claim 1 or 2, wherein the unlabeled data are data obtained from the same or a related application, or synthetic data generated from a data distribution of the labeled training data. [4] Method according to any one of claims 1 to 3, wherein the optimization algorithm is an evolutionary optimization or a Bayesian optimization. [5] Method according to any one of claims 1 to 4, wherein the difference is a Kullback-Leibler divergence or a mean square error or a hard label difference. [6] Method according to any one of claims 1 to 5, wherein the unlabeled data have been filtered, wherein the filtering is carried out by selecting data points of the unlabeled data set, wherein the confidence of the supermodel (1) is higher than a predefined threshold. [7] Method according to any one of claims 1 to 6, wherein the supermodel and the submodels are trained to be classifiers (60) for classifying sensor signals, wherein after training the classifier (60) is configured to receive a sensor signal (S) comprising data from a sensor (30), and determine an input signal (x), in particular an image, which depends on the sensor signal (S), and feed the input signal (x) into the classifier (60) to obtain an output signal (y) that characterizes a classification of the input signal (x). [8] Method according to any one of claims 1 to 7, wherein a submodel is selected from the Pareto-optimal submodels based on a predefined criterion and the selected submodel is used to provide an actuator control signal (A) for controlling an actuator (10), in particular by determining the actuator control signal (A) as a function of an output of the submodel, in particular as a function of the output signal (y). [9] Method according to claim 8, wherein the actuator (10) controls an at least partially autonomous robot (100) or a vehicle or manufacturing machine (200) or an access control system (300). [10] Computer program designed to cause a computer to execute the method according to any one of claims 1 to 9, including all its steps, when the computer program is executed by a processor (45, 145). [11] Machine-readable storage medium (46, 146) on which a computer program according to claim 10 is stored. [12] System designed to perform the method according to any one of claims 1 to 9.
