Method, apparatus and article of manufacture to generate hardware aware machine learning model architecture for multiple domains without training

Abstract

A technical solution for generating a hardware aware machine learning (ML) model architecture for multiple fields without training is disclosed. An example apparatus includes at least one memory, machine-readable instructions, and processor circuitry to instantiate or execute the machine-readable instructions. An example processor circuit is to generate a plurality of candidate architectures for an ML model based on target hardware to be used to execute the ML model and search spaces corresponding to a plurality of domains. Further, the example processor circuit is to calculate respective synthetic scores for the plurality of candidate architectures, the respective synthetic scores based on the respective delay scores for the plurality of candidate architectures. The example processor circuit is also to select an architecture for the ML model from a plurality of candidate architectures of the ML model, the selected architecture corresponding to a synthetic score associated with the selected architecture that meets a criterion.

Description

Technical Field

[0001] The present disclosure relates generally to machine learning, and more particularly to methods, apparatus, and articles of manufacture for generating hardware-aware machine learning model architectures for multiple domains without requiring training. Background Art

[0002] Machine learning models (e.g., neural networks) are useful tools that have proven their worth in solving complex problems in pattern recognition, natural language processing, automatic speech recognition, and more. For example, neural networks operate using artificial neurons arranged in layers that process data from an input layer to an output layer, applying weighted values ​​to the data as it is processed. These weighted values ​​are determined during the training process. The number of layers in a neural network corresponds to the depth of the network, with more layers resulting in a deeper network. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] Figure 1 is a network diagram including an example model generation controller.

[0004] Figure 2 yes Figure 1 A block diagram of a model generation controller for generating and evaluating one or more candidate architectures for one or more machine learning models to determine one or more architectures for one or more machine learning models that meet one or more criteria (e.g., highest score, best architecture, most suitable architecture, etc.).

[0005] Figure 3 It is shown Figure 1 and / or Figure 2 A table of sample configuration files for the model generation controller.

[0006] Figure 4 is a block diagram of an example transformer-based supernet framework of an example supernet disclosed herein.

[0007] Figure 5 Is Figure 1 and / or Figure 2 A visual representation of an example pipeline that a model generation controller executes to generate a desired (e.g., optimal, best, etc.) architecture for a machine learning model.

[0008] Figure 6 is a flowchart representing example machine-readable instructions and / or example operations that may be executed and / or instantiated by example processor circuits Figure 1 and / or Figure 2 The model generation controller to generate the machine learning model architecture.

[0009] Figure 7is a flowchart representing example machine-readable instructions and / or example operations that may be executed and / or instantiated by example processor circuits Figure 1 and / or Figure 2 The model generation controller is used to calculate the composite score of the candidate machine learning model architectures.

[0010] Figure 8 Shows Figure 1 and / or Figure 2 Graphical illustration of the performance comparison of our model generation controller with other neural architecture search methods.

[0011] Fig. 9 is a block diagram of an example processor platform including processor circuitry configured to perform Figure 6 and / or Figure 7 Example machine readable instructions and / or example operations to implement Figure 1 and / or Figure 2 The model generates the controller.

[0012] Fig.10 yes Fig. 9 A block diagram of an example implementation of a processor circuit.

[0013] Fig.11 yes Fig. 9 A block diagram of another example implementation of a processor circuit.

[0014] Fig.12 is a block diagram of an example software distribution platform (e.g., one or more servers) for distributing software (e.g., with Figure 6 and / or Figure 7 The software corresponding to example machine-readable instructions of the present invention is distributed to client devices associated with end-users and / or consumers (e.g., for licensing, sale, and / or use), retailers (e.g., for sale, resale, licensing, and / or sublicensing), and / or original equipment manufacturers (OEMs) (e.g., for inclusion in products to be distributed to, for example, retailers and / or other end-users such as direct purchasing customers).

[0015] In general, the same reference numerals will be used throughout the drawings (one or more) and the accompanying written description to refer to the same or similar parts. The drawings are not to scale. As used herein, a connection reference (e.g., attached, coupled, connected, joined) may include intermediate members between the elements referenced by the connection reference and / or relative movement between these elements, unless otherwise indicated. Therefore, a connection reference does not necessarily imply that two elements are directly connected and / or are in a fixed relationship to each other.

[0016] Unless otherwise specifically stated, descriptors such as "first," "second," "third," etc., are used herein without importing or otherwise indicating any priority, physical order, arrangement in a list, and / or meaning of being ordered in any manner, but are merely used as labels and / or arbitrary names to distinguish elements for ease of understanding of the disclosed examples. In some examples, the descriptor "first" may be used to refer to an element in the detailed description, while the same element may be referred to in the claims with a different descriptor, for example, "second" or "third." In this case, it should be understood that such descriptors are only used to clearly identify those elements that, for example, may share the same name.

[0017] As used herein, the phrase "communicating with..." (including variations thereof) encompasses direct communication and / or indirect communication through one or more intermediate components, without requiring direct physical (e.g., wired) communication and / or continuous communication, but also includes selective communication at periodic intervals, predetermined intervals, non-periodic intervals, and / or one-time events.

[0018] As used herein, a "processor circuit" is defined as including (i) one or more special-purpose electrical circuits that are constructed to perform (one or more) specific operations and include one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors), and / or (ii) one or more general-purpose semiconductor-based electrical circuits that are programmable with instructions to perform specific operations and include one or more semiconductor-based logic devices (e.g., electrical hardware implemented by one or more transistors). Examples of processor circuits include programmable microprocessors, field programmable gate arrays (FPGAs) (which can instantiate instructions), central processor units (CPUs), graphics processor units (GPUs), digital signal processors (DSPs), XPUs, or microcontrollers and integrated circuits, such as application-specific integrated circuits (ASICs). For example, an XPU can be implemented by a heterogeneous computing system that includes multiple types of processor circuits (e.g., one or more FPGAs, one or more CPUs, one or more GPUs, one or more DSPs, one or more DSPs, etc., and / or combinations thereof), and (one or more) application programming interfaces (APIs) that can assign (one or more) computing tasks to any (one or more) processor circuits of the multiple types that are best suited to perform the (one or more) computing tasks. In some examples, an ASIC is referred to as an application specific integrated circuit. DETAILED DESCRIPTION

[0019] Artificial intelligence (AI) (including machine learning (ML), deep learning (DL) and / or other artificial machine-driven logic) enables a machine (e.g., a computer, logic circuit, etc.) to process input data using a model to generate outputs based on patterns and / or associations that the model previously learned through a training process. For example, a model can be trained with data to recognize patterns and / or associations, and follow such patterns and / or associations when processing input data so that other (one or more) inputs produce (one or more) outputs consistent with the recognized patterns and / or associations.

[0020] In general, implementing an ML / AI system involves two phases, a learning / training phase and an inference phase. In the learning / training phase, a training algorithm is used to train the model to operate according to patterns and / or associations based on, for example, training data. In general, a model includes internal parameters that guide how input data is converted into output data, for example, through a series of nodes and connections within the model to convert input data into output data. In addition, hyperparameters are used as part of the training process to control how learning is performed (e.g., learning rate, number of layers to be used in a machine learning model, etc.). Hyperparameters are defined as training parameters that are determined before initiating the training process.

[0021] Different types of training can be performed based on the type and / or expected output of the ML / AI model. For example, supervised training uses inputs and corresponding expected (e.g., labeled) outputs to select parameters for the ML / AI model that reduce model error (e.g., by iteratively selecting combinations of parameters). As used herein, a label refers to the expected output of a machine learning model (e.g., a classification, an expected output value, etc.). Alternatively, unsupervised training (e.g., for deep learning, machine learning subsets, etc.) involves inferring patterns from inputs to select parameters for an ML / AI model (e.g., without the aid of expected (e.g., labeled) outputs).

[0022] Once trained, the deployed model can be operated in the inference phase to process data. In the inference phase, the data to be analyzed (e.g., real-time data) is input to the model, and the model executes to create an output. This inference phase can be thought of as the AI ​​"thinking" based on what it has learned from training to generate output (e.g., by executing the model to apply the learned patterns and / or associations to real-time data). In some examples, the input data is pre-processed before being used as input to the machine learning model. In addition, in some examples, the output data can be post-processed after being generated by the AI ​​model to convert the output into a useful result (e.g., a data display, instructions to be executed by the machine, etc.).

[0023] In some examples, the output of the deployed model can be captured and provided as feedback. By analyzing the feedback, the accuracy of the deployed model can be determined. If the feedback indicates that the accuracy of the deployed model is less than a threshold or other criteria, the feedback can be used to trigger training of an updated model along with an updated training data set, hyperparameters, etc. to generate an updated deployed model.

[0024] As described above, for example, a neural network operates using artificial neurons arranged in layers that process data from an input layer to an output layer and apply weighted values ​​to the data during the processing of the data. Models can be developed and / or trained to operate in different fields. Example fields include the fields of computer vision (CV), natural language processing (NLP), and recommendation systems (RecSys), among others. Typically, in order to develop a machine learning model, a human expert (e.g., an engineer) adjusts various aspects of the model until the human expert achieves the desired (e.g., optimal) model. For example, a human expert can adjust the model architecture and / or hyperparameters of a model so that the model achieves optimal performance on a given task.

[0025] Automated machine learning (ML) is a field of machine learning that aims to automate the process of developing a desired (e.g., best, optimal, etc.) model. Since models developed by neural architecture search (NAS) often outperform models developed by humans, NAS procedures have become an increasingly popular automated ML method. Typically, NAS procedures search the space of available model architectures and the space of available hyperparameter combinations to identify the best combination of model architectures and / or hyperparameters for a given task.

[0026] Although NAS is a promising approach, such programs are usually limited to a single domain. That is, NAS programs can develop high-performance models for tasks in one domain, but not for tasks in other domains. For example, NAS programs for convolutional neural networks (CNNs) are well suited for CV tasks, while NAS programs for recurrent neural networks (RNNs) are well suited for NLP tasks. In addition, developing NAS programs for a specific domain requires specific domain knowledge to construct a unique search space for (one or more) target tasks. Therefore, it is difficult to apply these domain-specific search spaces and model architectures to other domains.

[0027] In addition, NAS programs typically require a large amount of computing resources. For example, the search space of NAS programs is typically large (e.g., containing ten billion possible architectures), and existing NAS programs typically require iterative training and evaluation of candidate architectures to determine whether the candidate architectures meet thresholds or other criteria. Such NAS programs are not suitable for general-purpose processor circuits (e.g., CPUs and some edge devices) because the training-based performance predictors used to evaluate candidate architectures require a large amount of computing power for iterative evaluation. In addition, the performance predictors are training-based and therefore require data for performance evaluation. This requirement for data complicates the computational burden of tasks with large data sets (e.g., on the order of tens of thousands of elements). Therefore, existing NAS programs are typically executed on accelerators (e.g., GPUs or dedicated circuits), but even so, NAS may require multiple accelerators to execute for hundreds or even thousands of days, depending on the complexity of the search space.

[0028] In addition, many NAS programs are hardware-agnostic. That is, many NAS programs do not consider the target hardware on which the model will be executed when developing the model's architecture. This hardware-agnostic approach makes deployment in different devices difficult, especially for those with limited resources, such as edge devices and mobile devices. Although some NAS programs consider the target hardware, doing so exacerbates the computational resource requirements of the NAS program. For example, determining a model architecture suitable for different target hardware requires a specific search space, which can further complicate the search.

[0029] Some NAS programs use a one-shot approach to solve the huge computational burden of NAS. Under this one-shot NAS approach, the NAS program selects a general model (e.g., a supernet) and selects a submodel from the general model for a specific target hardware. In this method, the supernet is implemented as a directed acyclic graph (DAG), where the subgraphs of the DAG represent candidate models, and a lightweight performance predictor is used rather than iterative training and evaluation to select the candidate models. As used herein, supernets and supernets can be used interchangeably. In addition, some NAS programs have implemented performance evaluation agents rather than performance predictors to further reduce the computational burden of evaluating candidate models. The agent can be implemented by measuring some inherent characteristics of the model without the need to execute the model for training and evaluation. Therefore, the agent is sometimes referred to as a zero-cost agent.

[0030] However, a limitation of zero-cost proxies is that they only consider one or two features of a model, rather than a complete list of features of the model. For example, some proxy-based methods use the expressiveness of the model as a proxy, while others use a combination of the diversity and saliency of the model as a proxy. Expressiveness, diversity, and saliency are discussed further below. This limited zero-cost model evaluation results in zero-cost proxy NAS methods that perform well for one task or domain, but poorly for others.

[0031] To overcome the limitations of NAS, one-shot NAS, and zero-cost proxy NAS methods, the examples disclosed herein include multi-model, hardware-aware, training-free NAS methods to build compact (e.g., low computational complexity) model architectures for target hardware. For example, the disclosed examples build compact neural network architectures directly from human-designed search spaces that are applicable to multiple domains and multiple model types. The examples disclosed herein utilize hardware-aware search strategies based on one or more thresholds (e.g., model parameter size budgets) to determine the desired (e.g., optimal, best, etc.) model architecture, and utilize overall hardware-aware training-free scores to evaluate the performance of candidate architectures, rather than training each candidate architecture and obtaining the associated accuracy. For example, the overall training-free score takes into account the expressiveness, complexity, saliency, diversity, and latency of the candidate architectures.

[0032] Figure 1 1 is a network diagram 100 including an example model generation controller 102. The example network diagram 100 includes an example model generation controller 102, an example network 104, and an example target hardware platform 106. Figure 1 In the example, the example model generation controller 102 , the example target hardware platform 106 , and / or one or more additional devices are communicatively coupled via the example network 104 .

[0033] exist Figure 1 In the example shown, the model generation controller 102 is implemented by a processor circuit. Figure 1 In the example of , the model generation controller 102 is a server that implements a machine learning model (e.g., a parent model) to generate an architecture for one or more child models based on information specific to a target hardware platform. Figure 1 In the example of , the model generation controller 102 trains one or more child models.

[0034] exist Figure 1 In the illustrated example of , the model generation controller 102 implements a multi-model, hardware-aware, training-free NAS to build a compact, lightweight model architecture for target hardware (e.g., target hardware platform 106). Figure 1In the example, the model generation controller 102 constructs a compact neural architecture directly from a manually designed search space that supports multiple domains. The example model generation controller 102 utilizes a hardware-aware search strategy to determine a desired (e.g., optimal, best, etc.) network. In addition, the example model generation controller 102 employs a zero-cost, no-training scoring technique to evaluate the performance of candidate network architectures, rather than training each candidate architecture and obtaining the corresponding accuracy and / or another model-specific target metric.

[0035] exist Figure 1 In the illustrated example, the model generation controller 102 builds child models (e.g., child networks) based on a human-designed search space that is unified across multiple domains, multiple model types, and a shared model framework. In the examples disclosed herein, the model generation controller 102 builds candidate child model architectures using a transformer-based model framework that is tuned according to one or more supernets corresponding to one or more domains supported by the model generation controller 102. In addition, the example model generation controller 102 develops candidate architectures for child models based on the target hardware that will be used to execute the child models and the one or more domains in which the child models will operate. Figure 1 In the example of , the model generation controller 102 implements a zero-cost, no-training NAS method to evaluate candidate architectures of the descendant model.

[0036] The model generation controller 102 may be implemented using many different types of machine learning techniques and / or machine learning architectures. In examples disclosed herein, one or more components of the model generation controller 102 (e.g., search engine circuitry) are implemented by a hardware-aware evolutionary algorithm executed and / or instantiated on a processor circuit. The use of a hardware-aware evolutionary algorithm enables the model generation controller 102 to focus the search for candidate architectures on a relevant portion of the search space while enforcing upper limits on the delays and parameters of the sampled architectures and rejecting candidate architectures that exceed the upper limits to accommodate the target hardware requirements. Other types of machine learning techniques may be used in addition or in lieu of the hardware-aware search algorithm, such as reinforcement learning, random search, Bayesian optimization, gradient optimization, and the like.

[0037] In the examples disclosed herein, one or more components (e.g., training circuits) of the model generation controller 102 train the offspring model using stochastic gradient descent. However, any other training algorithm may be used in addition or alternatively. In the examples disclosed herein, training is performed until one or more thresholds are met. For example, the model generation controller 102 is trained to meet a threshold corresponding to the requirements of the hardware that implements the model generation controller 102 during reasoning. Additionally or alternatively, the model generation controller 102 is trained to meet a threshold corresponding to the requirements (e.g., accuracy) of the target hardware of the candidate architecture of the offspring model. In the examples disclosed herein, training is performed at a central server of the developer of the model generation controller 102. Training is performed using hyperparameters that control how learning is performed (e.g., learning rate, number of layers to be used in the machine learning model, etc.). In the examples disclosed herein, hyperparameters control the weights attributed to the characteristics of the candidate architecture when calculating the zero-cost, no-training score of the candidate architecture. Such hyperparameters are selected, for example, by the developer of the model generation controller 102.

[0038] Training is performed using training data. In the examples disclosed herein, the training data is derived from publicly available data sets. Since supervised training is used, the training data is labeled. Labels are applied to the training data by humans. In some examples, the training data is preprocessed to, for example, identify labels, reformat the training data into a format supported by the model generation controller 102, normalize the training data, etc. In some examples, the training data is subdivided into a training data set and a validation data set.

[0039] Once training is complete, the model is deployed as an executable construct that processes inputs and provides outputs based on the nodes and connected networks defined in the model. The model can be stored at a central server and provided as a service or for download. For example, the model generation controller 102 can provide one or more services and / or products to end users. In a service-based implementation, the model generation controller 102 can provide one or more trained models (e.g., candidate models) for download, a hosted network interface to access the model generation controller 102, etc. In a product-based implementation, the model generation controller 102 can provide a software package that implements the functions of the model generation controller 102. In this way, the end user can implement the model generation controller 102 locally (e.g., at the target hardware platform 106). In some examples, the model generation controller 102 can provide plug-ins compatible with ML development programs (e.g., TensorFlow, Keras, etc.) for end users. In such an example, the plug-in implements the functions of the model generation controller 102.

[0040] exist Figure 1In the example shown, the network 104 is the Internet. However, the example network 104 can be implemented using any suitable wired and / or wireless network(s), including, for example, one or more data buses, one or more local area networks (LANs), one or more wireless LANs, one or more cellular networks, one or more private networks, one or more public networks, etc. In additional or alternative examples, the network 104 is an enterprise network (e.g., within an enterprise, company, etc.), a home network, etc. The example network 104 enables the model generation controller 102 and the target hardware platform 106 to communicate.

[0041] exist Figure 1 In the example shown, the target hardware platform 106 is implemented by a laptop computer. In additional or alternative examples, the target hardware platform 106 may be implemented by a mobile phone, a tablet computer, a desktop computer, a server, etc. In some examples, the target hardware platform 106 may be implemented by a processor circuit, analog circuit(s), digital circuit(s), logic circuit(s), programmable processor(s), programmable microcontroller(s), GPU(s), DSP(s), ASIC(s), programmable logic device(s) (PLD), and / or field programmable logic device(s) (FPLD) (e.g., FPGA).

[0042] exist Figure 1 In the example shown, the target hardware platform 106 can subscribe to and / or otherwise purchase (one or more) products from the model generation controller 102 to access trained ML models optimized for the target hardware platform 106. For example, the target hardware platform 106 can access the trained ML model by downloading the model from the model generation controller 102, accessing a web interface hosted by the model generation controller 102 and / or another device, etc. In some examples, the target hardware platform 106 can install a plug-in to the ML training application. In such an example, the plug-in implements the model generation controller 102. In additional or alternative examples, the target hardware platform 106 can download a software application to implement the model generation controller 102.

[0043] Figure 2 yes Figure 1 A block diagram of a model generation controller 102 is provided for generating one or more candidate architectures for one or more ML models and training the one or more ML models. Figure 2 The model generation controller 102 may be instantiated (e.g., have an instance of it created, have it exist for any length of time, materialize, realize, etc.) by a processor circuit (e.g., a central processing unit executing instructions). Additionally or alternatively, Figure 2The model generation controller 102 may be instantiated (e.g., create an instance of, cause it to exist for any length of time, materialize, realize, etc.) by an ASIC or FPGA configured to perform operations corresponding to instructions (e.g., operations corresponding to instructions). It should be understood that Figure 2 Some or all of the circuits may thus be instantiated at the same or different times. Some or all of the circuits may be instantiated, for example, in one or more threads that execute simultaneously on the hardware and / or serially on the hardware. Furthermore, in some examples, Figure 2 Some or all of the circuits may be implemented in the following manner: a microprocessor circuit executes instructions to implement one or more virtual machines and / or containers.

[0044] exist Figure 2 In the example shown, the model generation controller 102 includes an example communication circuit 202, an example search engine circuit 204, an example predictor circuit 206, an example training circuit 208, and an example data storage device 210. Figure 2 In the example of the present invention, any of the communication circuit 202, the search engine circuit 204, the predictor circuit 206, the training circuit 208, and / or the data storage device 210 can communicate via the example communication bus 212. In the examples disclosed herein, the communication bus 212 can be implemented using any suitable wired and / or wireless communication. In additional or alternative examples, the communication bus 212 includes software, machine-readable instructions, and / or a communication protocol by which information is passed between the communication circuit 202, the search engine circuit 204, the predictor circuit 206, the training circuit 208, and / or the data storage device 210.

[0045] exist Figure 2 In the example shown, the communication circuit 202 interfaces with the network 104. For example, the communication circuit 202 receives one or more inputs indicating (e.g., at least one input indicating) a domain in which the ML model will operate and / or a target hardware on which the ML model will be executed. After the model generation controller 102 generates the architecture of the ML model and trains the ML model, the communication circuit 202 transmits the trained ML model to a client device (e.g., the target hardware platform 106).

[0046] In some examples, the communication circuit 202 receives one or more configuration files that indicate a search space of the ML model for one or more domains and / or one or more supernets applicable to the ML model. For example, a first supernet of the ML model corresponds to a first domain and a second supernet of the ML model corresponds to a second domain. In such an example, the communication circuit 202 forwards the configuration file(s) to the search engine circuit 204. In some examples, the configuration file(s) are stored in the data storage device 210 after receipt. In additional or alternative examples, the configuration file(s) are preloaded into the data storage device 210. In some examples, the communication circuit 202 is configured to execute communication instructions and / or is configured to execute, for example Figure 6 The flowchart represents the operation of a processor circuit to instantiate.

[0047] In some examples, model generation controller 102 includes means for communication. For example, means for communication can be implemented by communication circuit 202. In some examples, communication circuit 202 can be implemented by a processor circuit (e.g., Fig. 9 For example, the communication circuit 202 may be instantiated by executing machine executable instructions (e.g., by at least Figure 6 The block 602 of the instruction implemented by Fig.10 In some examples, the communication circuit 202 can be instantiated by a hardware logic circuit, which can be implemented by an ASIC, an XPU, or Fig.11 The communication circuit 202 may be implemented by an FPGA circuit 1100 configured to perform operations corresponding to machine-readable instructions. Additionally or alternatively, the communication circuit 202 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the communication circuit 202 may be implemented by at least one or more hardware circuits (e.g., processor circuits, discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, XPUs, comparators, operational amplifiers (op-amps), logic circuits, etc.), which are configured to execute some or all machine-readable instructions and / or execute some or all operations corresponding to machine-readable instructions without executing software or firmware, but other structures are equally applicable.

[0048] Figure 3 It is shown Figure 1 and / or Figure 2 The table of an example configuration file 300 of the model generation controller 102. In some examples, the configuration file 300 can be formatted as a Yet Another Markup Language (YAML) file. In additional or alternative examples, the configuration file 300 can be formatted in any manner. Figure 3 In the example of , the configuration file 300 includes an example domain portion 302, an example search space portion 304, and an example supernet portion 306. In the examples disclosed herein, the search space of the ML model is a unified search space applicable to multiple domains and / or multiple model types. For example, the search space can be applicable to the computer vision field, the natural language processing field, and the recommendation system field. In this way, the search space can be unified across multiple domains. Figure 3 In the example of , the domain portion 302 indicates that the configuration set forth in the configuration file 300 supports both the CV domain and the NLP domain, and is therefore unified across the CV and NLP domains.

[0049] Furthermore, the example unified search space disclosed herein is applicable to multiple model types. For example, the search space includes building blocks for CNNs, transformer models, multi-layer perceptron (MLP) models, etc. In this way, in addition to being unified across multiple domains, the disclosed search space is also unified across multiple model types. Providing a unified search space that supports multiple model types allows the model generation controller 102 to support different domains. Figure 3 In the example of , the search space portion 304 indicates that the search space of the configuration file 300 supports supernets that utilize transformer models and / or MLP models. Example transformer layers in the supported supernets can range from 8 to 16, where each transformer layer has 3 to 12 multi-head self-attention (MSA) sublayers; query, key, and value matrices ranging in size from 192 to 768 elements with a step size of 64 elements; embedding dimensions of 192 elements, 216 elements, 240 elements, 320 elements, 384 elements, 448 elements, 528 elements, 576 elements, and 624 elements; and three MLP layers. The three MLP layers include: a first MLP layer having a size range of 128 to 768 elements with a step size of 16 elements; a second MLP layer having a size range of 128 to 768 elements with a step size of 16 elements; and a third MLP layer having a size range of 128 to 3072 elements with a step size of 32 elements. In other examples, the parameters of the supernet layers may be different. For example, developers may change the parameters included in the configuration file based on their application.

[0050] In the examples disclosed herein, a unified search space is developed to include optimization building blocks (e.g., ML operations) that have been incorporated into the search space based on prior knowledge (e.g., typical properties of architectures that are well suited to the task). By including the optimization building blocks and leveraging prior knowledge, the examples disclosed herein reduce the search space size. Figure 3 In the example, the supernet portion 306 indicates Figure 3For example, the supernet portion 306 includes an example first supernet 306A corresponding to the CV domain and an example second supernet 306B corresponding to the NLP domain.

[0051] Figure 4 is a block diagram of an example supernet framework 400 of an example supernet disclosed herein. Figure 4 In the example of , the supernet framework 400 represents each layer of the supernet and is applicable to both the first supernet 306A (which is applicable to the CV domain) and the second supernet 306B (which is applicable to the NLP domain). Therefore, in addition to being unified across multiple domains and multiple model types, the disclosed search space is unified across a single supernet framework. Figure 4 In the example of , the supernet framework 400 is a unified converter. Therefore, Figure 3 and Figure 4 The search space is based on the search space of the unified transformer.

[0052] exist Figure 4 In the illustrated example, each transformer layer of the example supernet framework 400 receives an example domain-specific embedding vector 402. The domain-specific embedding vector 402 is fed into an example value matrix 404, an example key matrix 406, and an example query matrix 408. The value matrix 404, the key matrix 406, and the query matrix 408 are fed into one or more example attention heads 410. For example, the number of attention heads included in the transformer layer can be specified in the configuration file 300. One or more outputs of the one or more attention heads 410 are fed into an example first MLP layer 412. The output of the first MLP layer 412 and the domain-specific embedding vector 402 are fed into an example first addition and normalization layer 414.

[0053] exist Figure 4 In the example shown, the output of the first addition and normalization layer 414 is fed into the example second MLP layer 416. The output of the second MLP layer 416 is fed into the example third MLP layer 418, and the output of the third MLP layer 418 is fed into the example second addition and normalization layer 420. The output of the second addition and normalization layer 420 is the example domain-specific output vector 422. The domain-specific output vector 422 can be fed into any subsequent transformer layer. In addition, in Figure 4 In the example of , the second MLP layer 416, the third MLP layer 418, and the second addition and normalization layer 420 form an internal feed-forward network for each transformer layer. In the examples disclosed herein, the number of MLP layers, the number of transformer layers, the number of attention heads, and the size of queries, keys, and values ​​included in the supernet can vary. For example, the number of MLP layers, the number of transformer layers, the number of attention heads, and the size of queries, keys, and values ​​can be specified in the configuration file 300.

[0054] Back to Figure 2 In response to the request to generate an ML model, the search engine circuit 204 selects a supernet for the ML model based on the domain in which the ML model will operate. The search engine circuit 204 selects a supernet from at least two supernets corresponding to each of the plurality of domains supported by the search space. From the selected supernets, the search engine circuit 204 generates candidate architectures for the ML model by searching the unified search space according to a search algorithm. In the examples disclosed herein, the search engine circuit 204 supports a plurality of search algorithms, some of which are combined with Figure 5 Have a discussion.

[0055] In this manner, the search engine circuitry 204 supports pluggable search algorithms (e.g., one or more search strategies that can be "plugged in" and "unplugged" from the search engine circuitry 204). Figure 2 In the example of , the search engine circuit 204 generates candidate architectures for the ML model by searching a unified search space (e.g., a hardware-aware search algorithm) based on the target hardware that will be used to execute the ML model. For example, pseudo code 1 shows a hardware-aware search algorithm with evolution (e.g., an evolutionary algorithm (EA)) that can be implemented by the model generation controller 102.

[0056] 1.F 0 =Initial_Architecture(K,S,T s ,T l )

[0057] 2. Topk = Φ

[0058] 3.for i=1:N do

[0059] 4.DE SCORE =DE SCORE (F i-1 )

[0060] 5. Topk = update_topk (F i-1 ,DE SCORE ,Topk)

[0061] 6.F mutation =Mutation(N m ,S,T s ,T l ,p,Topk)

[0062] 7.F crossover =Crossover(N c ,S,T s ,T l ,Topk)

[0063] 8.F i =F mutation ∪F crossover

[0064] 9.end

[0065] 10. Return the topk with the best DE SCORE Network architecture

[0066] Pseudocode 1

[0067] exist Figure 2 In the example shown, the search engine circuit 204 selects the candidate architecture based on the number of candidate architectures sampled from the supernet (K), the search space (S), the parameter threshold of the target hardware (T s ) and the latency threshold of the target hardware (T l ) Generate initial candidate architectures for ML models from the supernet (F 0 )(for example, line 1 of Pseudocode 1). Figure 2 In the example of FIG. 2 , the search engine circuit 204 initializes a variable (Topk) in the data storage device 210 to store the composite scores and the corresponding candidate architectures (e.g., line 2 of pseudo code 2). For example, the variable can be a multi-dimensional matrix or tensor (Φ) formatted to store the composite scores and the corresponding candidate architectures. In the examples disclosed herein, the variable can be referred to as a candidate architecture tracking variable or a composite score array. After generating scores for the initial candidate architectures, the search engine circuit 204 uses the composite scores (DE SCORE ) and the corresponding candidate architecture (F i-1 ) updates the candidate architecture tracking variable (e.g., the integrated score array) (e.g., line 5 of pseudocode 1).

[0068] exist Figure 2 In the example shown, the search engine circuit 204 generates mutations (F mutation )(e.g., line 6 of pseudo code 1). For example, a mutation corresponds to the search engine circuit 204 randomly (e.g., pseudo-randomly) changing one or more building blocks (e.g., ML operations) of a previous candidate architecture. Generating mutations increases the diversity of candidate architectures and allows the search engine circuit 204 to avoid reaching a local optimal architecture but not a global optimal architecture. Figure 2 In the example of FIG. 1 , the search engine circuit 204 generates a mutation based on a mutation size parameter (N m ), the search space of the ML model (S), the parameter threshold of the target hardware (T s ), the latency threshold of the target hardware (T l ), mutation probability (p), and candidate architecture tracking variables (Topk) (e.g., comprehensive score group) to generate mutations.

[0069] exist Figure 2 In the example shown, the search engine circuit 204 generates a crossover (F crossover )(e.g., line 7 of pseudo code 1). For example, a crossover corresponds to the search engine circuit 204 exchanging one or more building blocks of a previous candidate architecture with other building blocks in a related subspace of the search space. Generating a crossover mixes the candidate architectures in the related subspace and allows the search engine circuit 204 to converge to a desired (e.g., optimal) architecture in the related subspace. Figure 2 In the example of FIG. 1 , the search engine circuit 204 generates a signal based on the crossover size parameter (N c ), the search space of the ML model (S), the parameter threshold of the target hardware (T s ), the latency threshold of the target hardware (T l ) and the candidate architecture tracking variables (Topk) (e.g., comprehensive score groups) generate crossovers.

[0070] exist Figure 2 In the example shown, the parameter threshold used by the search engine circuit 204 corresponds to a number of parameters (e.g., weights) that the ML model is allowed to have when deployed to the target hardware. The example latency threshold used by the search engine circuit 204 corresponds to the duration that the ML model is allowed to complete inference on the target hardware. By implementing the parameter threshold and the latency threshold, the search engine circuit 204 filters out (e.g., focuses the search on) candidate architectures that are suitable for the target hardware (e.g., the target hardware platform 106) in a coarse-grained manner.

[0071] exist Figure 2 In the example shown, the search engine circuit 204 generates additional candidate architectures (F i )(for example, line 8 of Pseudocode 1). Figure 2 In an example of , search engine circuitry 204 generates additional candidate architectures as a union of mutations and crossovers. In additional or alternative examples, search engine circuitry 204 generates additional candidate architectures as any other combination of mutations and crossovers.

[0072] Subsequently, the search engine circuit 204 updates the candidate architecture tracking variable (e.g., the composite score group) with the composite score of the candidate architecture for a predefined number (N) of iterations and generates additional candidate architectures for the ML model from the supernet. In response to completing the predefined number of iterations, the search engine circuit 204 returns the architecture of the ML model from the supernet (e.g., line 10 of pseudo code 1). The returned architecture corresponds to the candidate score of the candidate architecture tracking variable (e.g., the composite score group) that meets the criteria. Figure 2In the example of , the criterion corresponds to the maximum composite score among the composite scores. In additional or alternative examples, other criteria can be used, for example, returning scores that meet (e.g., are equal to or greater than) a threshold. In some examples, search engine circuitry 204 is configured to execute search engine instructions and / or to execute, for example Figure 6 The flowchart represents the operation of a processor circuit to instantiate.

[0073] In some examples, model generation controller 102 includes means for searching. For example, the means for searching can be implemented by search engine circuit 204. In some examples, search engine circuit 204 can be implemented by processor circuit (e.g., Fig. 9 For example, the search engine circuit 204 may be instantiated by executing machine executable instructions (e.g., by at least Figure 6 604, 606, 610, 612, 614, 616, 618 and 620 of Fig.10 In some examples, the search engine circuit 204 can be instantiated by a hardware logic circuit, which can be implemented by an ASIC, an XPU, or Fig.11 The search engine circuit 204 is implemented by an FPGA circuit 1100 that is configured to perform operations corresponding to machine-readable instructions. Additionally or alternatively, the search engine circuit 204 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the search engine circuit 204 may be implemented by at least one or more hardware circuits (e.g., processor circuits, discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, XPUs, comparators, operational amplifiers (op-amps), logic circuits, etc.), which are configured to execute some or all machine-readable instructions and / or execute some or all operations corresponding to machine-readable instructions without executing software or firmware, but other structures are equally applicable.

[0074] exist Figure 2 In the example shown, the predictor circuit 206 calculates a composite score for the candidate architectures generated during the execution of the search algorithm by the search engine circuit 204 (e.g., line 4 of pseudo code 1). For example, the predictor circuit 206 utilizes a composite zero-cost proxy to predict the accuracy of the candidate architectures, rather than fully training and validating an ML model with the candidate architectures. In this way, the composite zero-cost proxy is a no-training score. Figure 2 In the example of , the predictor circuit 206 calculates the composite score of the candidate architecture as a combination of the expressiveness of the candidate architecture, the complexity of the candidate architecture, the diversity of the candidate architecture, the salience of the candidate architecture, and the latency of the candidate architecture. Figure 2 In the example of , the predictor circuit 206 calculates the composite score of the candidate architecture according to the following equation 1:

[0075] DE SCORE =(α 1 D EXP +α 2 D COM +α 3 D DIV +α 4 D SAL )·D LAT

[0076] Equation 1

[0077] In Equation 1, DE SCORE represents the comprehensive score of the candidate architecture, D EXP represents the expressiveness score of the candidate architecture, D COM Denotes the complexity score of the candidate architecture, D DIV Denotes the diversity score of the candidate architecture, D SAL represents the significance score of the candidate architecture, D LAT represents the latency score of the candidate architecture. In addition, α 1 , α 2 , α 3 and α 4 are the hyperparameters for expressiveness, complexity, diversity, and significance, respectively. In order to calculate the comprehensive score (DE SCORE ), the predictor circuit 206 calculates the product of each hyperparameter and the expressiveness score, complexity score, diversity score, and significance score of the candidate architecture. In addition, the predictor circuit 206 calculates the comprehensive score (DE SCORE ) is the product of the delay score and the sum of the products. Then, the search engine circuit 204 updates the search algorithm based on the comprehensive score (eg, lines 4-8 of pseudo code 1).

[0078] In the example of Equation 1, the hyperparameter α 1 , α 2 , α 3 and α 4 Controls whether the corresponding scores will be used for different model types that may be developed by the model generation controller 102. For example, the complexity score may be more relevant to evaluating the MLP layer, while the significance score and diversity score may be more relevant to evaluating the transformer layer. Figure 2 In the example of FIG. 1 , the predictor circuit 206 uses the hyperparameter α 1 , α 2 , α 3 and α 4 In the examples disclosed herein, the binary value is adjusted by the developer of the model generation controller 102. In additional or alternative examples, the hyperparameter α 1, α 2 , α 3 and α 4 Can be implemented as a continuous value from zero to one, for example.

[0079] exist Figure 2 In the example shown, the predictor circuit 206 uses expressivity as a measure of the ability of the candidate architecture to approximate complex functions. For example, the more expressive the architecture is (e.g., the larger the expressivity score), the more efficiently the architecture can fit the training data. For example, for a CNN including multiple layers, each layer having a convolution operation followed by a rectified linear unit (ReLU) activation, expressivity can be measured by the Gaussian complexity of the candidate architecture. The predictor circuit 206 calculates the Gaussian complexity according to Equation 2 below.

[0080] D EXP =log(E x,δ ‖f(x)-f(x+γδ)‖) Equation 2

[0081] In equation 2, D EXP represents the expressiveness score of the candidate architecture, E x,δ represents the expected value of a function of independent variables having the input (x) of the ML model and the parameters (δ) of the ML model, ‖*‖ represents the Euclidean norm, f(*) represents the pre-global average pooling feature map of the candidate architecture (F(*)), and γ represents the coefficient of the parameters (δ) of the ML model. In implementing Equation 2, the predictor circuit 206 samples the input (x) and parameters (δ) of the ML model using random (e.g., pseudo-random) numbers from a standard normal distribution (e.g., with a mean of zero and a variance of one). Figure 2 In the example of , predictor circuit 206 solves Equation 2 with the coefficient (γ) of parameter (δ) set to 0.01.

[0082] exist Figure 2 In the example shown, the predictor circuit 206 uses complexity as a measure of the ability of the candidate architecture to be optimized by gradient descent. For example, the architecture of the ML model can control the efficiency with which gradient information can flow through the ML model. The complexity of the candidate architecture can be measured by a neural tangent kernel (NTK) score. The predictor circuit 206 calculates the NTK score according to Equation 3 below.

[0083]

[0084]

[0085] In Equation 3, D COM represents the complexity score of the candidate architecture (e.g., NTK score), E xrepresents the mean of some function of the independent variables with the input (x) of the ML model, represents NTK, λ max represents the maximum eigenvalue of NTK, λ min represents the minimum eigenvalue of NTK. Figure 2 In the example of FIG. 1 , the predictor circuit 206 calculates NTK according to the following equation 4

[0086]

[0087] In Equation 4, J(x) represents the Jacobian matrix (Jacobian) found at point x. When implementing Equations 3 and 4, the predictor circuit 206 samples the input (x) using random (e.g., pseudo-random) numbers from a standard normal distribution. Figure 2 In the example of , the predictor circuit 206 utilizes diversity as a measure of the ability of the MSA layer of the candidate architecture to capture different properties of the input embedded in different subspaces. For example, the higher the rank of the parameter (e.g., weight) matrix of the MSA layer, the higher the diversity score (e.g., the more diverse the information that the MSA layer can capture). Figure 2 In the example of , predictor circuit 206 calculates diversity according to Equation 5 below.

[0088]

[0089] In Equation 5, D DIV represents the diversity score of the candidate architecture, L represents the loss function of the ML model with parameters W of the MSA layer, ‖*‖ nuc represents the nuclear norm of the parameter (e.g., weight) matrix (W), and ⊙ represents the Hadamard product. When implementing Equation 5, the predictor circuit 206 calculates the sum of the Hadamard products of the nuclear norm of the weights and the nuclear norm of the corresponding gradient of the loss function for all weights of the MSA layer of the candidate architecture. Figure 2 In the example of , the predictor circuit 206 utilizes significance as a measure of the number of important parameters (e.g., weights) in the MLP layer of the candidate architecture. For example, significance is a common metric for gradient-based scores used to prune parameters (e.g., to meet a parameter threshold) and is retained in the hidden units and layers of the candidate architecture. Figure 2 In the example of , the predictor circuit 206 calculates the significance according to Equation 6 below.

[0090]

[0091] In Equation 6, D SALrepresents the significance score of the MLP layer of the candidate architecture, L represents the loss function of the ML model with the parameters W of the MLP layer, and ⊙ represents the Hadamard product. When implementing Equation 6, the predictor circuit 206 adds the Hadamard product of the corresponding gradient of the weight and the loss function for all weights of all MLP layers of the candidate architecture. Figure 2 In the example of , the predictor circuit 206 utilizes latency as a measure of whether the candidate architecture can meet the latency requirements of the target hardware. In this way, latency can be used to restrict NAS to candidate architectures that will operate satisfactorily on the target hardware. Figure 2 In the example of , predictor circuit 206 calculates the delay according to Equation 7 below.

[0092]

[0093] In Equation 7, D LAT represents the latency score of the candidate architecture, T BATCH represents the expected duration of inference for executing the ML model on a batch of input data on the target hardware, and β represents a hyperparameter that controls the weight attributed to the latency score in the composite score (e.g., the ratio of the latency score to other scores). In the examples disclosed in this article, β is a continuous variable ranging between zero and one. By default, Figure 2 In the example of , the predictor circuit 206 sets β to one to solve Equation 7. In Equation 7, 1 is added to the denominator to avoid the expected duration of the inference (T BATCH ) very briefly with inaccurate latency scores.

[0094] In the examples disclosed herein, equations 2, 3, 5, 6, and 7 are example equations used by the predictor circuit 206 to calculate expressivity, complexity, diversity, significance, and delay, respectively. In additional or alternative examples, the predictor circuit 206 calculates expressivity, complexity, diversity, significance, and delay in other ways. If the predictor circuit 206 calculates expressivity, complexity, diversity, significance, and delay using alternative equations, the developer of the predictor circuit 206 should adjust the equations for their application. For example, the developer of the predictor circuit 206 can adjust the alternative equations based on the domains supported by the model generation controller 102 (e.g., CV domain, NLP domain, RecSys domain, etc.).

[0095] Pseudo-code 2 illustrates an example composite score algorithm that may be implemented by predictor circuit 206 .

[0096] 1. By Initialize all neurons in F(*)

[0097] 2. Sampling

[0098] 3. Calculate the network expressiveness score D EXP , significance score D SAL ,

[0099] Network complexity score D COM , network diversity score D DIV

[0100] 4. Calculate the batch delay time and calculate the delay score D LAT

[0101] 5. Calculate DE SCORE =(α 1 D EXP +α 2 D COM +α 3 D DIV+ α 4 D SAL )

[0102] D LAT

[0103] 6. Return to DE SCORE (F(*))

[0104] Pseudocode 2

[0105] exist Figure 2 In the example of , at line 1 of pseudo code 2, the predictor circuit 206 initializes all neurons of the candidate architecture F(*). Figure 2 In the example of , the normal distribution has a mean of zero and a variance of one. At line 2 of pseudo code 2, the predictor circuit 206 samples the inputs (x) and parameters (δ) of the candidate architecture F(*) using random (e.g., pseudo-random) numbers from the normal distribution. At line 3 of pseudo code 2, the predictor circuit 206 calculates the expressiveness score (D) of the candidate architecture according to equations 2, 3, 4, 5, 6, and 7 described above. EXP ), the complexity score of the candidate architecture (D COM ), the diversity score of the candidate architecture (D DIV ) and the significance score of the candidate architecture (D SAL In line 4 of pseudo code 2, the predictor circuit 206 calculates the expected duration (e.g., batch latency (T )) of executing inference of the candidate architecture on a batch of input data on the target hardware. BATCH )). In addition, in line 4 of pseudo code 2, the predictor circuit 206 calculates the batch delay time (T BATCH ) Calculate the latency score (D) of the candidate architecture LAT ).

[0106] exist Figure 2In the example shown, at line 5 of pseudo code 2, the predictor circuit 206 calculates the composite score (DE) of the candidate architecture according to equation 1 described above. SCORE For example, the predictor circuit 206 uses the delay fraction as the inverse of the sum of the other fractions. Therefore, when the batch delay time (T BATCH ) is small (e.g., close to zero), the overall untrained comprehensive score (DE SCORE ) will be larger. In the examples disclosed herein, the predictor circuit 206 calculates the composite score (DE) in only a few forward inferences. SCORE ), rather than iteratively training candidate architectures. Thus, examples disclosed herein evaluate candidate architectures in an extremely fast (e.g., fewer computation cycles than other NAS methods), lightweight (e.g., requiring relatively few computational resources than other NAS methods), and data-free (e.g., requiring no training data) manner.

[0107] In some examples, model generation controller 102 includes means for predicting. For example, means for predicting can be implemented by predictor circuit 206. In some examples, predictor circuit 206 can be implemented by a processor circuit (e.g., Fig. 9 For example, the predictor circuit 206 may be instantiated by executing machine executable instructions (e.g., by at least Figure 6 Block 608 and / or at least Figure 7 702, 704, 706, 708, 710, 712, 714, 716 and 718 of Fig.10 In some examples, the predictor circuit 206 can be instantiated by a hardware logic circuit, which can be implemented by an ASIC, an XPU, or Fig.11 The predictor circuit 206 may be implemented by an FPGA circuit 1100 configured to perform operations corresponding to the machine-readable instructions. Additionally or alternatively, the predictor circuit 206 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the predictor circuit 206 may be implemented by at least one or more hardware circuits (e.g., processor circuits, discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, XPUs, comparators, operational amplifiers (op-amps), logic circuits, etc.), which are configured to execute some or all machine-readable instructions and / or execute some or all operations corresponding to the machine-readable instructions without executing software or firmware, but other structures are equally applicable.

[0108] exist Figure 2In the illustrated example, the search engine circuit 204 and the predictor circuit 206 operate to use a search algorithm (e.g., pseudo code 1) and a no-training composite score to maximize a composite score to determine a desired (e.g., optimal, best, etc.) architecture. As described above, the composite score incorporates the expected latency of the candidate architectures to evaluate the candidate architectures. In this way, the search engine circuit 204 and the predictor circuit 206 filter out (e.g., focus the search on) candidate architectures that are suitable for the target hardware (e.g., the target hardware platform 106) in a fine-grained manner. In some examples, the predictor circuit 206 is configured to execute predictor instructions and / or to execute, for example Figure 6 and / or Figure 7 The flowchart represents the operation of a processor circuit to instantiate.

[0109] exist Figure 2 In the example shown, the training circuit 208 implements a training algorithm to train an ML model having a structure returned by the search algorithm executed by the search engine circuit 204. Figure 2 In the example disclosed herein, stochastic gradient descent is used to train the ML model generated by the search algorithm executed by the search engine circuit 204. However, any other training algorithm may be used in addition or alternatively. In the example disclosed herein, training is performed for a threshold number of epochs to converge to a set of parameters that provide the most accurate reasoning for the ML model. In the example disclosed herein, training is performed at a central server of the developer of the model generation controller 102. Training is performed using hyperparameters that control how learning is performed (e.g., learning rate, number of layers to be used in the machine learning model, etc.). In the example disclosed herein, hyperparameters control the learning rate of the ML model, the number of layers of the ML model, the width of the layer, pruning, etc. Such hyperparameters are selected, for example, by the developer of the model generation controller 102. In some examples, retraining may be performed. For example, after an end user downloads the ML model from the model generation controller 102, the end user may adjust the model for their application.

[0110] exist Figure 2 In the example shown, the training circuit 208 trains the ML model generated by the search algorithm using the training data. In the examples disclosed herein, the training data is derived from a publicly available collection, such as the CIFAR-10 dataset, the CIFAR-100 dataset, and / or a public text repository, such as Yelp reviews, IMDB reviews, and WordNet, etc. Since supervised training is used, the training data is labeled. The labels are applied to the training data by humans. In some examples, the training data is preprocessed to, for example, identify labels, reformat the training data into a format supported by the ML model, normalize the training data, etc. In some examples, the training data is subdivided into a training data set and a validation data set.

[0111] Once training is complete, the ML model is deployed as an executable construct that processes inputs and provides outputs based on the network of nodes and connections defined in the ML model. The example trained models disclosed herein are high-performance and lightweight models (e.g., providing accurate reasoning without consuming excessive computing resources). In some examples, the trained ML model is stored at a central server and provided as a service to end users. In such examples, the trained ML model can then be executed by the central server (e.g., the model generation controller 102) based on inputs received from client devices. In additional or alternative examples, the end user can download the trained ML model to the client device. In such an example, the trained ML model can then be executed by the client device (e.g., the target hardware platform 106). In some examples, the training circuit 208 is configured to execute training instructions and / or is configured to execute, for example Figure 6 The flowchart represents the operation of a processor circuit to instantiate.

[0112] In some examples, model generation controller 102 includes means for training. For example, means for training can be implemented by training circuit 208. In some examples, training circuit 208 can be implemented by a processor circuit (e.g., Fig. 9 For example, the training circuit 208 may be instantiated by executing machine executable instructions (e.g., by at least Figure 6 The instructions implemented by blocks 622 and 624 of Fig.10 In some examples, the training circuit 208 may be instantiated by a hardware logic circuit, which may be implemented by an ASIC, an XPU, or Fig.11 The training circuit 208 is implemented by an FPGA circuit 1100 that is configured to perform operations corresponding to machine-readable instructions. Additionally or alternatively, the training circuit 208 may be instantiated by any other combination of hardware, software, and / or firmware. For example, the training circuit 208 may be implemented by at least one or more hardware circuits (e.g., processor circuits, discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, XPUs, comparators, operational amplifiers (op-amps), logic circuits, etc.), which are configured to execute some or all machine-readable instructions and / or execute some or all operations corresponding to machine-readable instructions without executing software or firmware, but other structures are equally applicable.

[0113] exist Figure 2In the example shown, the data storage device 210 is configured to store data. For example, the data storage device 210 may store one or more files indicating one or more configuration files, one or more search algorithms, training data, one or more trained models, information related to the target hardware, etc. The example configuration file includes one or more search spaces and one or more supernets. The example search space includes optimizing ML operations and / or optimizing ML components. Such ML operations include convolution operations, operations specific to state-of-the-art (SOTA) models (e.g., ResNet blocks, DLRM blocks, BERT blocks, etc.). Such ML components include kernel size, number of filters, linear layer dimensions, etc. Example information related to the target hardware includes parameter thresholds and delay thresholds. In some examples, the information related to the target hardware may be lower-level information, such as the memory capacity of the target hardware and the memory bandwidth of the target hardware, etc. In such an example, the model generation controller 102 calculates parameter thresholds and delay thresholds based on the lower-level information.

[0114] exist Figure 2 In the example shown, the data storage device 210 can be implemented by volatile memory (e.g., synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), RAMBUS dynamic random access memory (RDRAM), etc.) and / or non-volatile memory (e.g., flash memory). The example data storage device 210 can be additionally or alternatively implemented by one or more double data rate (DDR) memories, such as DDR, DDR2, DDR3, DDR4, mobile DDR (mDDR), etc. The example data storage device 210 can be additionally or alternatively implemented by one or more mass storage devices, such as (one or more) hard disk drives, (one or more) optical disk drives, (one or more) digital versatile disk drives, (one or more) solid state disk drives, etc. Although the data storage device 210 is shown as a single database in the example shown, the data storage device 210 can be implemented by any number and / or (one or more) types of databases. In addition, the data stored in the data storage device 210 can be in any data format, such as binary data, comma-delimited data, tab-delimited data, structured query language (SQL) structures, etc.

[0115] Figure 5 Is Figure 1 and / or Figure 2 A visual representation of an example pipeline 500 that the model generation controller 102 of ML executes to generate a desired (e.g., optimal, best, etc.) architecture for an ML model. Figure 5 In the example, the search engine circuit 204 accesses the example search space 502 of the ML model For example, search engine circuitry 204 accesses search space 502 from data storage device 210 and / or receives an indication of search space 502 from a configuration file. Figure 5 In the example, the search space 502 is unified across multiple domains, multiple model types, and a single supernet framework. For example, the supported domains include the CV domain, the NLP domain, and the RecSys domain. The model types supported by the example include CNN, transformer models, and MLP models. The example supernet framework includes a transformer framework.

[0116] exist Figure 5 In the example shown, the search space 502 is unified across the CV domain and the NLP domain. Figure 5 The search space 502 is unified across the Transformer model and the MLP model. Figure 5 In the example of , the search space 502 is unified across the converter framework. Figure 5 In the example of , the search engine circuit 204 identifies the domain in which the ML model will operate and the target hardware on which the ML model will be executed. For example, in addition to the configuration file, the model generation controller 102 receives one or more indications (e.g., from an end user) indicating the domain in which the ML model will operate and the target hardware on which the ML model will be executed.

[0117] exist Figure 5 In the example shown, based on the indicated domain, the search engine circuit 204 selects a supernet for the ML model from the at least two supernets. As described above, the at least two supernets for the ML model correspond to the respective domains supported by the search space 502. In addition, the at least two supernets corresponding to the respective domains share a supernet framework. Figure 5 In the example of , the search engine circuit 204 executes the example pluggable search algorithm 506 to search for a candidate architecture (A) in the search space 502. For example, the pluggable search algorithm 506 can be implemented by the example evolutionary algorithm 506A, the example reinforcement learning algorithm 506B, the example hardware-aware algorithm 506C, and / or the example pruning algorithm 506D.

[0118] exist Figure 5 In the illustrated example, the search engine circuit 204 executes the example hardware-aware evolutionary algorithm as described above in conjunction with pseudo code 1. Thus, the search engine circuit 204 generates a plurality of candidate architectures for the ML model from the supernet, the plurality of candidate architectures being based on the target hardware on which the ML model will be executed and the search space corresponding to the plurality of domains. In addition, the example predictor circuit 206 calculates a plurality of composite scores for the plurality of candidate architectures, each composite score being based at least on a respective latency score for the plurality of candidate architectures. For example, the composite score comprises an example expressiveness score 508, an example complexity score 510, an example diversity score 512, an example significance score 514, and an example latency score 516. In Figure 5 In the example of , the search engine circuit 204 and the predictor circuit 206 operate to return a framework of the ML model from the supernet, where the framework corresponds to a composite score among multiple composite scores that satisfies the criteria (e.g., a maximum composite score among the multiple composite scores).

[0119] exist Figure 5 In the illustrated example of , the search engine circuit 204 and the predictor circuit 206 return an example compact neural architecture 518 that requires relatively fewer computational resources to generate than other NAS methods. Subsequently, the training circuit 208 trains the compact neural architecture 518 based on the training data retrieved from the data storage device 210 to generate an example trained compact model 520. Subsequently, the trained compact model 520 can be deployed to a client device (e.g., the target hardware platform 106).

[0120] Although in Figure 2 The implementation is shown in the figure Figure 1 The model generation controller 102 is an example of a method, but Figure 2 One or more of the components, processes, and / or devices shown may be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other manner. Figure 1 and / or Figure 2 The example communication circuit 202, the example search engine circuit 204, the example predictor circuit 206, the example training circuit 208, and / or more generally the example model generation controller 102 may be implemented in hardware alone or in combination with software and / or firmware. Thus, for example, Figure 1 and / or Figure 2 Any of the example communication circuit 202, the example search engine circuit 204, the example predictor circuit 206, the example training circuit 208, and / or more generally the example model generation controller 102 may be implemented by a processor circuit, (one or more) analog circuits, (one or more) digital circuits, (one or more) logic circuits, (one or more) programmable processors, (one or more) programmable microcontrollers, (one or more) graphics processing units (GPUs), (one or more) digital signal processors (DSPs), (one or more) application specific integrated circuits ASICs, (one or more) programmable logic devices (PLDs), and / or (one or more) field programmable logic devices (FPLDs) (e.g., field programmable gate arrays (FPGAs)). In addition, Figure 1 An example model generation controller 102 may include, in addition to Figure 2 In addition to or in place of those shown in Figure 2 The present invention may include one or more elements, processes, and / or devices of those shown in the drawings, and / or may include more than one of any or all of the illustrated elements, processes, and devices.

[0121] exist Figure 6 and Figure 7 A flowchart is shown in FIG. 1 , which represents instructions that may be executed to configure a processor circuit (eg, machine-readable instructions that cause the processor circuit to implement Figure 1 and / or Figure 2 The model generates example machine-readable instructions for the controller 102. The machine-readable instructions may be one or more executable programs or (one or more) parts of executable programs for execution by a processor circuit, for example, as described below in conjunction with Fig. 9 The processor circuit 912 shown in the example processor platform 900 discussed below and / or in conjunction with Fig.10 and / or Fig.11 Example processor circuit discussed. The program may be embodied as software stored on one or more non-transitory computer-readable storage media associated with a processor circuit located in one or more hardware devices, such as a compact disk (CD), a floppy disk, a hard disk drive (HDD), a solid-state drive (SSD), a digital versatile disk (DVD), a Blu-ray disk, a volatile memory (e.g., any type of random access memory (RAM), etc.), or a non-volatile memory (e.g., an electrically erasable programmable read-only memory (EEPROM), FLASH memory, HDD, SSD, etc.), but the entire program and / or its parts may alternatively be executed by one or more hardware devices other than the processor circuit, and / or embodied in firmware or dedicated hardware. Machine-readable instructions may be distributed across multiple hardware devices and / or executed by two or more hardware devices (e.g., a server and a client hardware device). For example, a client hardware device may be implemented by an endpoint client hardware device (e.g., a hardware device associated with a user) or an intermediate client hardware device (e.g., a radio access network (RAN) gateway that facilitates communication between a server and an endpoint client hardware device). Similarly, a non-transitory computer-readable storage medium may include one or more media located in one or more hardware devices. In addition, although the reference Figure 6 and Figure 7The flowchart shown describes an example program, but many other methods of implementing the example model generation controller 102 may be used instead. For example, the order of execution of the blocks may be changed, and / or some of the blocks described may be changed, eliminated, or combined. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., processor circuits, discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) that are constructed to perform the corresponding operations without executing software or firmware. The processor circuits may be distributed at different network locations and / or located locally on one or more hardware devices (e.g., single-core processors (e.g., single-core central processing units (CPUs)), multi-core processors in a single machine (e.g., multi-core CPUs, XPUs, etc.), multiple processors distributed on multiple servers in a server rack, multiple processors distributed on one or more server racks, CPUs and / or FPGAs located in the same package (e.g., the same integrated circuit (IC) package or two or more separate housings, etc.)).

[0122] The machine-readable instructions described herein may be stored in one or more of a compressed format, an encrypted format, a segmented format, a compiled format, an executable format, a packaged format, etc. The machine-readable instructions as described herein may be stored as data or data structures (e.g., as part of instructions, codes, code representations, etc.) that can be used to create, manufacture, and / or generate machine-executable instructions. For example, the machine-readable instructions may be segmented and stored on one or more storage devices and / or computing devices (e.g., servers) located at the same or different locations of a network or a collection of networks (e.g., in the cloud, on edge devices, etc.). The machine-readable instructions may require one or more of installation, modification, adaptation, update, combination, supplementation, configuration, decryption, decompression, decapsulation, distribution, reassignment, compilation, etc., so that they can be directly readable, interpreted, and / or executable by a computing device and / or other machine. For example, the machine-readable instructions may be stored as multiple parts that are individually compressed, encrypted, and / or stored on separate computing devices, where the parts, when decrypted, decompressed, and / or combined, form a set of machine-executable instructions that implement one or more operations that can together form a program as described herein.

[0123] In another example, the machine-readable instructions may be stored in a state in which they can be read by a processor circuit, but require the addition of a library (e.g., a dynamic link library (DLL)), a software development kit (SDK), an application programming interface (API), etc.) in order to execute the machine-readable instructions on a particular computing device or other device. In another example, the machine-readable instructions may need to be configured (e.g., store settings, input data, record network addresses, etc.) before the machine-readable instructions and / or corresponding program(s) can be executed in whole or in part. Thus, a machine-readable medium as used herein may include machine-readable instructions and / or program(s) regardless of the particular format or state of the machine-readable instructions and / or program(s) when stored or otherwise at rest or in transmission.

[0124] The machine-readable instructions described herein may be represented by any past, current, or future instruction language, scripting language, programming language, etc. For example, any of the following languages ​​may be used to represent machine-readable instructions: C, C++, Java, C#, Perl, Python, JavaScript, Hypertext Markup Language (HTML), Structured Query Language (SQL), Swift, etc.

[0125] As mentioned above, Figure 6 and / or Figure 7The example operations of can be implemented using executable instructions (e.g., computer and / or machine readable instructions) stored on one or more non-transitory computer and / or machine readable media, such as optical storage devices, magnetic storage devices, HDDs, flash memory, read-only memory (ROM), CDs, DVDs, caches, any type of RAM, registers, and / or any other storage device or storage disk in which information is stored for any duration (e.g., storage for a longer period of time, permanent storage, temporary storage, for temporary buffering, and / or for caching of information). As used herein, the terms non-transitory computer readable medium, non-transitory computer readable storage medium, non-transitory machine readable medium, and non-transitory machine readable storage medium are expressly defined to include any type of computer readable storage device and / or storage disk, and do not include propagating signals and do not include transmission media. As used herein, the terms "computer readable storage device" and "machine readable storage device" are defined to include any physical (mechanical and / or electrical) structure that stores information, but do not include propagating signals and do not include transmission media. Examples of computer-readable storage devices and machine-readable storage devices include any type of random access memory, any type of read-only memory, solid-state memory, flash memory, optical disks, magnetic disks, disk drives, and / or redundant arrays of independent disks (RAID) systems. As used herein, the term "device" refers to a physical structure, such as a mechanical and / or electrical device, hardware, and / or circuit, that may or may not be configured and / or manufactured to perform computer-readable instructions, machine-readable instructions, etc., using computer-readable instructions, machine-readable instructions, etc.

[0126] "Include" and "comprising" (and all forms and tenses thereof) are used as opening terms herein. Thus, whenever a claim employs any form of "include" or "comprising" (e.g., includes, comprises, has, etc.) as a preamble or used in any type of claim recitation, it will be understood that additional elements, terms, etc. may be present without falling outside the scope of the corresponding claim or recitation. As used herein, when the phrase "at least" is used as a transitional term, such as in the preamble of a claim, it is opening in the same way that the terms "include" and "comprising" are opening. The term "and / or" when used, for example, in the form of A, B, and / or C, refers to any combination or subset of A, B, and C, for example, (1) A alone, (2) B alone, (3) C alone, (4) A and B, (5) A and C, (6) B and C, or 7) A and B and C. As used herein in the context of describing structures, components, items, objects, and / or things, the phrase “at least one of A and B” is intended to refer to implementations that include any of the following: (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, as used herein in the context of describing structures, components, items, objects, and / or things, the phrase “at least one of A or B” is intended to refer to implementations that include any of the following: (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. As used herein in the context of describing the performance and / or execution of processes, instructions, actions, activities, and / or steps, the phrase “at least one of A and B” is intended to refer to implementations that include any of the following: (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, as used herein in the context of describing the performance or execution of a process, instruction, action, activity, and / or step, the phrase "at least one of A or B" is intended to refer to an implementation that includes any of the following: (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.

[0127] As used herein, singular references (e.g., "one," "first," "second," etc.) do not exclude a plurality. As used herein, the term "an" object refers to one or more of the object. The terms "one," "one or more," and "at least one" are used interchangeably herein. In addition, although listed separately, multiple means, elements, or method actions may be implemented by, for example, the same entity or object. In addition, although individual features may be included in different examples or claims, they may be combined, and inclusion in different examples or claims does not imply that a combination of features is not feasible and / or disadvantageous.

[0128] Figure 6is a flowchart representing example machine-readable instructions and / or example operations 600 that may be executed and / or instantiated by an example processor circuit. Figure 1 and / or Figure 2 The model generation controller 102 is used to generate a machine learning model architecture. Figure 6 The machine readable instructions and / or operations 600 begin at block 602, where the communication circuit 202 receives a Figure 1 The model generation controller 102 may be associated with one or more inputs of a query. For example, the query may be a request from a client device to generate a trained ML model to operate in a specific domain on specific hardware (e.g., target hardware platform 106).

[0129] exist Figure 6 In the example shown, one or more inputs indicate the domain in which the ML model will operate and the target hardware on which the ML model will be executed. The one or more inputs indicating the target hardware may identify a model of the target hardware and / or some other identification information of the target hardware. As described below, in some examples, the search engine circuit 204 may use the identification information to determine a parameter threshold of the target hardware and / or a delay threshold of the target hardware (e.g., by accessing a lookup table (LUT), accessing the data storage device 210, etc.). In some examples, the one or more inputs indicating the target hardware may include a parameter threshold of the target hardware and a delay threshold of the target hardware.

[0130] exist Figure 6 In the example shown, at block 604, based on the domain, the search engine circuit 204 selects a supernet for the ML model from at least two supernets. For example, as described above, the communication circuit 202 may receive a configuration file with a query and / or the configuration file may be stored in the data storage device 210. The example configuration file indicates supported domains, a unified search space, and supernets for the supported domains. At block 605, the model generation controller 102 implements an example search algorithm to develop an architecture of the ML model from the supernet. Example search algorithms include evolutionary algorithms, reinforcement learning algorithms, hardware-aware algorithms, and / or pruning algorithms. Figure 6 In the example of , the model generation controller 102 implements an example hardware-aware evolutionary algorithm at block 605 .

[0131] exist Figure 6In the example shown, at block 606, the search engine circuit 204 generates candidate architectures of the ML model from the supernet based on the target hardware and the search space corresponding to the multiple domains supported by the search engine circuit 204. For example, the search engine circuit 204 accesses a unified search space supporting the CV domain and the NLP domain to generate the candidate architectures. At block 608, the predictor circuit 206 calculates a comprehensive score of the candidate architecture based on at least the latency score of the candidate architecture. For example, the predictor circuit 206 implements pseudo code 2 as described above to calculate the comprehensive score of the candidate architecture. In combination Figure 7 Example implementations of block 608 are discussed.

[0132] exist Figure 6 In the example shown, at block 610, the search engine circuit 204 updates the composite score group using the composite score corresponding to the candidate architecture. The search engine circuit 204 also updates the group using the candidate architecture. At block 612, the search engine circuit 204 generates mutations of the candidate architecture. For example, the search engine circuit 204 generates mutations based on a mutation size parameter, a search space of the ML model, a parameter threshold of the target hardware, a delay threshold of the target hardware, a mutation probability, and a composite score group and a corresponding candidate architecture. In some examples, the search engine circuit 204 can access information identifying the target hardware (e.g., a model, memory and / or computing power of the target hardware, etc.), and use this information to determine a parameter threshold and / or a delay threshold (e.g., by looking up such a threshold in the data storage device 210, accessing a LUT, etc.). In additional or alternative examples, the parameter threshold and / or the delay threshold are included in one or more inputs received by the communication circuit 202.

[0133] exist Figure 6 In the illustrated example of FIG. 6 , at block 614, the search engine circuit 204 generates a crossover of candidate architectures. For example, the search engine circuit 204 generates the crossover based on a crossover size parameter, a search space of the ML model, a parameter threshold of the target hardware, a latency threshold of the target hardware, and a comprehensive score group and corresponding candidate architectures. At block 616, the search engine circuit 204 generates additional candidate architectures of the ML model as a combination of mutations and crossovers. For example, the search engine circuit 204 generates additional candidate architectures of the ML model as a combination of mutations and crossovers.

[0134] exist Figure 6 In the example shown, at block 618, the search engine circuitry 204 determines whether there are additional search iterations to be performed in the search algorithm. In response to the search engine circuitry 204 determining that there are additional search iterations to be performed (block 618: yes), the machine-readable instructions and / or operations 600 return to block 608. In response to the search engine circuitry 204 determining that there are no additional search iterations to be performed (block 618: no), the machine-readable instructions and / or operations 600 continue to block 620.

[0135] exist Figure 6 In the example shown, at block 620, the search engine circuit 204 returns the architecture of the ML model from the supernet. For example, the architecture corresponds to the composite score in the composite score group that satisfies the criteria. Figure 6 In an example of , the criterion is that the composite score is the maximum composite score in the composite score group. At block 622, using the architecture of the ML model, the training circuitry 208 trains the ML model on the training data set. At block 624, the training circuitry 208 deploys the trained ML model. For example, the training circuitry 208 instructs the communication circuitry 202 to transmit (e.g., cause transmission) the trained ML model to a client device, such as the target hardware platform 106. In some examples, the training circuitry 208 causes the trained ML model to be stored in the data storage device 210. After block 624, the machine-readable instructions and / or operations 600 terminate.

[0136] Figure 7 is a flow chart representing example machine-readable instructions and / or example operations 608 that may be executed and / or instantiated by example processor circuitry to implement Figure 1 and / or Figure 2 The model generation controller 102 is used to calculate the comprehensive score of the candidate machine learning model architecture. Figure 7 The machine readable instructions and / or operations 608 of begin at block 702, where the predictor circuit 206 initializes the neurons of the candidate architecture according to a distribution. For example, the predictor circuit 206 initializes the neurons of the candidate architecture with a normal distribution having a mean of zero and a variance of one.

[0137] exist Figure 7 In the example shown, at block 704, the predictor circuit 206 samples the distribution to generate an input matrix and a parameter matrix for the candidate architecture. For example, the predictor circuit 206 samples the inputs and parameters using random (e.g., pseudo-random) numbers from a normal distribution. At block 706, the predictor circuit 206 calculates the expressiveness score for the candidate architecture. For example, the predictor circuit 206 implements Equation 2 as described above to calculate the expressiveness score.

[0138] exist Figure 7 In the example shown, at block 708, the predictor circuit 206 calculates a complexity score for the candidate architecture. For example, the predictor circuit 206 implements equations 3 and 4 as described above to calculate the complexity score. At block 710, the predictor circuit 206 calculates a diversity score for the candidate architecture. For example, the predictor circuit 206 implements equation 5 as described above to calculate the diversity score. At block 712, the predictor circuit 206 calculates a significance score for the candidate architecture. For example, the predictor circuit 206 implements equation 6 as described above to calculate the significance score.

[0139] exist Figure 7 In the example shown, at block 714, the predictor circuit 206 calculates a batch latency time for the candidate architecture. For example, the batch latency time corresponds to the expected duration of performing inference of the candidate architecture on a batch of input data on the target hardware. At block 716, the predictor circuit 206 calculates a latency score for the candidate architecture based on the batch latency time. For example, the predictor circuit 206 implements equation 7 as described above to calculate the latency score. At block 718, the predictor circuit 206 calculates a composite score for the candidate architecture based on the expressiveness score, the complexity score, the diversity score, the significance score, and the latency score. For example, the predictor circuit 206 implements equation 1 as described above to calculate the composite score.

[0140] Figure 8 Shows Figure 1 and / or Figure 2 A graphical illustration 800 is provided for comparing the performance of the model generation controller 102 with other neural architecture search methods. The graphical illustration 800 includes an example first graphical illustration 802, an example second graphical illustration 804, and an example third graphical illustration 806. Figure 8 In the example of , graphical illustration 800 illustrates the number of floating point operations per second (FLOPS) of the model, the number of parameters of the model, and the performance indicators of the model (accuracy, training time, inference time, etc.). The disclosed examples have been tested and verified for multiple example applications including CV and NLP fields.

[0141] exist Figure 8 In the example shown, the first graphical illustration 802 shows a comparison between the model (DE-Net) generated by the model generation controller 102 and the general ResNet 101 model. Figure 8 In the example, the DE-Net model and the general ResNet101 model are Xeon Gold 6252 server. As shown in the first graphical illustration 802, compared with the general ResNet101 model on the CIFAR-10 dataset, the DE-Net model generated by the model generation controller 102 has 70 times fewer parameters, 8.4 times less training time, and 2% higher accuracy.

[0142] exist Figure 8In the example shown, the second graphical illustration 804 shows a comparison of the improvements achieved by the DE-Net model generated by the model generation controller 102 and the models generated using two SOTA zero-cost NAS methods (Zen-NAS and Synflow) relative to the general ResNet101 model. The second graphical illustration 804 illustrates the results of the model trained on the CIFAR-10 dataset. As shown in the second graphical illustration 804, the number of parameters of the DE-Net model generated by the model generation controller 102 is reduced by 70 times and the training time is reduced by 8.4 times compared to the general ResNet101 model. The reduction in the number of parameters and training time achieved by the DE-Net model generated by the model generation controller 102 is greater than the reduction in the number of parameters and training time achieved by the models developed by Zen-NAS or Synflow. In addition, the accuracy improvement achieved by the DE-Net model generated by the model generation controller 102 is greater than the accuracy improvement achieved by the models developed by Zen-NAS or Synflow. The FLOPS reduction achieved by the DE-Net model generated by the model generation controller 102 is comparable to the FLOPS reduction achieved by the models developed by Zen-NAS and Synflow.

[0143] exist Figure 8 In the example shown, the third graphical illustration 806 shows a comparison between (1) the model generation controller 102 and the DE-Net model generated by the model generation controller 102 and (2) the SOTA NAS method, the Autoformer, and the model developed by the Autoformer. The third graphical illustration 806 illustrates the results of the transformer-based DE-Net model trained on the CIFAR-10 dataset. As shown in the third graphical illustration 806, the model generation controller 102 speeds up the search time by 38.36 times compared to the Autoformer, and speeds up the training time by 1.62 times compared to the Autoformer. The DE-Net model generated by the model generation controller 102 speeds up the inference by 1.79 times compared to the model developed by the Autoformer, and reduces the number of parameters by 3.86 times compared to the model developed by the Autoformer.

[0144] Fig. 9 is constructed to execute and / or instantiate Figure 6 Machine readable instructions and / or operations 600 and / or Figure 7 Machine readable instructions and / or operations 608 to implement Figure 11 and / or 2. The processor platform 900 may be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), a mobile device (e.g., a cellular phone, a smart phone, an iPad, etc.). TM , a tablet device such as a iPod, a personal digital assistant (PDA), an Internet appliance, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a game console, a personal video recorder, a set-top box, headphones (e.g., an augmented reality (AR) headset, a virtual reality (VR) headset, etc.) or other wearable device, or any other type of computing device.

[0145] The processor platform 900 of the illustrated example includes a processor circuit 912. The processor circuit 912 of the illustrated example is hardware. For example, the processor circuit 912 can be implemented by one or more integrated circuits, logic circuits, FPGAs, microprocessors, CPUs, GPUs, DSPs, and / or microcontrollers from any desired series or manufacturer. The processor circuit 912 can be implemented by one or more semiconductor-based (e.g., silicon-based) devices. In this example, the processor circuit 912 implements the example search engine circuit 204, the example predictor circuit 206, and the example training circuit 208.

[0146] The processor circuit 912 of the illustrated example includes a local memory 913 (e.g., cache, registers, etc.). The processor circuit 912 of the illustrated example communicates with a main memory including a volatile memory 914 and a non-volatile memory 916 via a bus 918. The volatile memory 914 may be a synchronous dynamic random access memory (SDRAM), a dynamic random access memory (DRAM), Dynamic Random Access Memory The non-volatile memory 916 may be implemented by flash memory and / or any other desired type of memory device. Access to the main memory 914 , 916 of the illustrated example is controlled by a memory controller 917 .

[0147] The processor platform 900 of the illustrated example further includes an interface circuit 920. The interface circuit 920 may be implemented by hardware according to any type of interface standard, for example, an Ethernet interface, a universal serial bus (uUSB) interface, interface, a near field communication (NFC) interface, a peripheral component interconnect (PCI) interface, and / or a peripheral component interconnect express (PCIe) interface.

[0148] In the illustrated example, one or more input devices 922 are connected to the interface circuit 920. The input device(s) 922 allow a user to enter data and / or commands into the processor circuit 912. The input device(s) 922 may be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, buttons, a mouse, a touch screen, a trackpad, a trackball, an isopoint device, and / or a voice recognition system.

[0149] One or more output devices 924 are also connected to the interface circuit 920 of the illustrated example. The output device(s) 924 may be implemented, for example, by a display device (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube (CRT) display, an in-place switching (IPS) display, a touch screen, etc.), a tactile output device, a printer, and / or a speaker. The interface circuit 920 of the illustrated example thus typically includes a graphics driver card, a graphics driver chip, and / or a graphics processor circuit, such as a GPU.

[0150] The interface circuitry 920 of the illustrated example also includes a communication device, such as a transmitter, a receiver, a transceiver, a modem, a residential gateway, a wireless access point, and / or a network interface to facilitate data exchange with an external machine (e.g., any kind of computing device) via a network 926. Communications can be made through, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a line-of-sight wireless system, a cellular telephone system, an optical connection, etc. In this example, the interface circuitry 920 implements the communication circuitry 202.

[0151] The processor platform 900 of the illustrated example also includes one or more mass storage devices 928 to store software and / or data. Examples of such mass storage devices 928 include magnetic storage devices, optical storage devices, floppy disk drives, HDDs, CDs, Blu-ray disk drives, redundant arrays of independent disks (RAID) systems, solid-state storage devices (e.g., flash memory devices and / or SSDs), and DVD drives. In this example, one or more mass storage devices 928 implement the data storage device 210.

[0152] Can be Figure 6 Machine readable instructions and / or operations 600 and / or Figure 7 The machine-readable instructions and / or machine-readable instructions 932 implemented by operation 608 may be stored in the mass storage device 928, in the volatile memory 914, in the non-volatile memory 916, and / or on a removable non-transitory computer-readable storage medium (e.g., a CD or DVD).

[0153] Fig.10 yes Fig. 9 912. In this example, Fig. 9 The processor circuit 912 is implemented by the microprocessor 1000. For example, the microprocessor 1000 may be a general-purpose microprocessor (eg, a general-purpose microprocessor circuit). The microprocessor 1000 executes Figure 6 and / or Figure 7 Some or all of the machine-readable instructions of a flowchart to effectively Figure 2 The circuits are instantiated as logic circuits to perform operations corresponding to these machine-readable instructions. In some such examples, Figure 2 The circuit is instantiated by the hardware circuit of the microprocessor 1000 in combination with the instructions. For example, the microprocessor 1000 can be implemented by a multi-core hardware circuit, such as a CPU, a DSP, a GPU, an XPU, etc. Although any number of example cores 1002 (e.g., 1 core) may be included, the microprocessor 1000 of this example is a multi-core semiconductor device including N cores. The cores 1002 of the microprocessor 1000 can operate independently or can cooperate to execute machine-readable instructions. For example, the machine code corresponding to a firmware program, an embedded software program, or a software program may be executed by one of the cores 1002, or may be executed by multiple cores in the cores 1002 at the same or different times. In some examples, the machine code corresponding to the firmware program, the embedded software program, or the software program is divided into threads and executed in parallel by two or more cores in the cores 1002. The software program may correspond to Figure 6 and / or Figure 7 The flowchart may represent part or all of the machine-readable instructions and / or operations.

[0154] The core 1002 can communicate via a first example bus 1004. In some examples, the first bus 1004 can be implemented by a communication bus to implement communications associated with one (or more) of the cores 1002. For example, the first bus 1004 can be implemented by at least one of an Inter-Integrated Circuit (IC) bus, a Serial Peripheral Interface (SPI) bus, a PCI bus, or a PCIe bus. Additionally or alternatively, the first bus 1004 can be implemented by any other type of computing or electrical bus. The core 1002 can obtain data, instructions, and / or signals from one or more external devices via an example interface circuit 1006. The core 1002 can output data, instructions, and / or signals to one or more external devices via the interface circuit 1006. While the core 1002 of this example includes an example local memory 1020 (e.g., a level 1 (L1) cache that can be partitioned into an L1 data cache and an L1 instruction cache), the microprocessor 1000 also includes an example shared memory 1010 (e.g., a level 2 (L2 cache)) that can be shared by the cores for high-speed access to data and / or instructions. Data and / or instructions can be transferred (e.g., shared) by writing to and / or reading from the shared memory 1010. The local memory 1020 and shared memory 1010 of each core 1002 can be a plurality of levels of cache memory and main memory (e.g., Fig. 9 Caches are part of a hierarchy of storage devices (main memory 914, 916). Typically, higher level memories in the hierarchy exhibit lower access times and have smaller storage capacities than lower level memories. Changes in the various levels of the cache hierarchy are managed (e.g., coordinated) by cache coherence policies.

[0155] Each core 1002 may be referred to as a CPU, DSP, GPU, etc., or any other type of hardware circuit. Each core 1002 includes a control unit circuit 1014, an arithmetic and logic (AL) circuit 1016 (sometimes referred to as an arithmetic and logic circuit, ALU, etc.), a plurality of registers 1018, a local memory 1020, and a second example bus 1022. Other structures may exist. For example, each core 1002 may include a vector unit circuit, a single instruction multiple data (SIMD) unit circuit, a load / store unit (LSU) circuit, a branch / jump unit circuit, a floating point unit (FPU) circuit, etc. The control unit circuit 1014 (e.g., a control circuit) includes a semiconductor-based circuit configured to control data movement (e.g., coordinate data movement) within the corresponding core 1002. The AL circuit 1016 includes a semiconductor-based circuit configured to perform one or more mathematical and / or logical operations on data within the corresponding core 1002. The AL circuit 1016 of some examples performs integer-based operations. In other examples, the AL circuit 1016 also performs floating-point operations. In still other examples, AL circuit 1016 may include a first AL circuit that performs integer-based operations and a second AL circuit that performs floating-point operations. In some examples, AL circuit 1016 may be referred to as an arithmetic logic unit (ALU). Register 1018 is a semiconductor-based structure used to store data and / or instructions, such as the results of one or more operations performed by AL circuit 1016 of the corresponding core 1002. For example, register 1018 may include (one or more) vector registers, (one or more) SIMD registers, (one or more) general registers, (one or more) flag registers, (one or more) segment registers, (one or more) machine-specific registers, (one or more) instruction pointer registers, (one or more) control registers, (one or more) debug registers, (one or more) memory management registers, (one or more) machine check registers, etc. Register 1018 may be as follows: Fig.10 The registers 1018 are shown arranged as a library. Alternatively, the registers 1018 may be organized in any other arrangement, format, or structure, including being distributed throughout the core 1002 to reduce access time. The second bus 1022 may be implemented by at least one of an I2C bus, an SPI bus, a PCI bus, or a PCIe bus.

[0156] Each core 1002 and / or more generally the microprocessor 1000 may include structures in addition to and / or in place of the structures shown and described above. For example, there may be one or more clock circuits, one or more power supplies, one or more power gates, one or more cache home agents (CHA), one or more converged / common mesh stops (CMS), one or more shifters (e.g., (one or more) barrel shifters), and / or other circuits. The microprocessor 1000 is a semiconductor device that is manufactured to include many interconnected transistors to implement the above structure in one or more integrated circuits (ICs) contained in one or more packages. The processor circuit may include and / or cooperate with one or more accelerators. In some examples, the accelerator is implemented by a logic circuit to perform certain tasks faster and / or more efficiently than a general-purpose processor. Examples of accelerators include ASICs and FPGAs, as discussed herein. A GPU or other programmable device may also be an accelerator. The accelerator may be on a board of a processor circuit, in the same chip package as the processor circuit, and / or in one or more packages separate from the processor circuit.

[0157] Fig.11 yes Fig. 9 1 is a block diagram of another example implementation of a processor circuit 912 of FIG. 1. In this example, the processor circuit 912 is implemented by an FPGA circuit 1100. For example, the FPGA circuit 1100 can be implemented by an FPGA. For example, the FPGA circuit 1100 can be used to perform, for example, Fig.10 The example microprocessor 1000 of the embodiment of the present invention executes the operations performed by the corresponding machine-readable instructions. However, once configured, the FPGA circuit 1100 instantiates the machine-readable instructions in hardware and can therefore generally execute the operations faster than a general-purpose microprocessor executes the corresponding software.

[0158] More specifically, Fig.10 A microprocessor 1000 (which is a general purpose device that can be programmed to perform Figure 6 and / or Figure 7 Compared to a flowchart representing some or all of the machine-readable instructions, but whose interconnections and logic circuits are fixed once they are manufactured, Fig.11 The example FPGA circuit 1100 includes interconnects and logic circuits that can be configured and / or interconnected in different ways after fabrication to instantiate, for example, Figure 6 and / or Figure 71100. Specifically, FPGA circuit 1100 can be considered as an array of logic gates, interconnects, and switches. The switches can be programmed to change how the logic gates are interconnected by the interconnects, effectively forming one or more dedicated logic circuits (unless and until FPGA circuit 1100 is reprogrammed). The configured logic circuits enable the logic gates to cooperate in different ways to perform different operations on the data received by the input circuits. These operations can correspond to Figure 6 and / or Figure 7 Thus, FPGA circuit 1100 can be constructed to effectively Figure 6 and / or Figure 7 Some or all of the machine-readable instructions of the flowchart of FIG. 1100 are instantiated as dedicated logic circuits to perform operations corresponding to these software instructions in a dedicated manner similar to an ASIC. Figure 6 and / or Figure 7 Some or all of the machine-readable instructions may correspond to operations faster than a general purpose microprocessor can perform those operations.

[0159] exist Fig.11 In the example of FPGA circuit 1100, FPGA circuit 1100 is configured to be programmed (and / or reprogrammed one or more times) by an end user via a hardware description language (HDL) (e.g., Verilog). Fig.11 FPGA circuit 1100 includes example input / output (I / O) circuit 1102 to obtain data from and / or output data to example configuration circuit 1104 and / or external hardware 1106. For example, configuration circuit 1104 can be implemented by an interface circuit that can obtain machine-readable instructions to configure FPGA circuit 1100 or (one or more) portions thereof. In some such examples, configuration circuit 1104 can obtain machine-readable instructions from a user, a machine (e.g., a hardware circuit (e.g., programmed or dedicated circuit) that can implement an artificial intelligence / machine learning (AI / ML) model to generate instructions). In some examples, external hardware 1106 can be implemented by external hardware circuits. For example, external hardware 1106 can be implemented by Fig.10 FPGA circuit 1100 also includes an array of example logic gate circuits 1108, a plurality of example configurable interconnects 1110, and example storage circuits 1112. Logic gate circuits 1108 and configurable interconnects 1110 can be configured to instantiate Figure 6 and / or Figure 7 One or more operations corresponding to at least some of the machine-readable instructions, and / or other desired operations. Fig.11The logic gate circuits 1108 shown are manufactured in groups or blocks. Each block includes a semiconductor-based electrical structure that can be configured into a logic circuit. In some examples, the electrical structure includes a logic gate (e.g., an AND gate, an OR gate, a NOR gate, etc.) that provides a basic building block for the logic circuit. There are electrically controllable switches (e.g., transistors) in each logic gate circuit 1108 so that the electrical structure and / or logic gate can be configured to form a circuit to perform the desired operation. The logic gate circuit 1108 may include other electrical structures, such as a lookup table (LUT), a register (e.g., a flip-flop or a latch), a multiplexer, etc.

[0160] The configurable interconnect 1110 of the illustrated example is a conductive path, trace, via, etc., which may include an electrically controllable switch (e.g., a transistor) whose state can be changed by programming (e.g., using an HDL instruction language) to activate or deactivate one or more connections between one or more logic gate circuits 1108 to program the desired logic circuit.

[0161] The storage circuit 1112 of the illustrated example is configured to store (one or more) results of one or more operations performed by corresponding logic gates. The storage circuit 1112 can be implemented by registers, etc. In the illustrated example, the storage circuit 1112 is distributed between the logic gate circuits 1108 to facilitate access and improve execution speed.

[0162] Fig.11 The example FPGA circuit 1100 also includes an example dedicated operation circuit 1114. In this example, the dedicated operation circuit 1114 includes a dedicated circuit 1116, which can be called to implement common functions to avoid the need to program these functions on site. Examples of such dedicated circuits 1116 include memory (e.g., DRAM) controller circuits, PCIe controller circuits, clock circuits, transceiver circuits, memories, and multiplier-accumulator circuits. Other types of dedicated circuits may exist. In some examples, the FPGA circuit 1100 may also include an example general-purpose programmable circuit 1118, for example, an example CPU 1120 and / or an example DSP 1122. Other general-purpose programmable circuits 1118 may exist in addition or alternatively, such as a GPU, XPU, etc., which may be programmed to perform other operations.

[0163] Although Fig.10 and Fig.11 Picture shows Fig. 9 These are two example implementations of the processor circuit 912, and many other implementations are also contemplated. For example, as described above, modern FPGA circuits may include an onboard CPU, such as, Fig.11 One or more example CPUs 1120. Therefore, Fig. 9 The processor circuit 912 may additionally be combined with Fig.10An example microprocessor 1000 and Fig.11 In some such hybrid examples, Figure 6 and / or Figure 7 The first portion of the machine-readable instructions represented by the flowchart of Fig.10 One or more cores 1002 execute, Figure 6 and / or Figure 7 The second portion of the machine-readable instructions represented by the flowchart of Fig.11 The FPGA circuit 1100 performs, and / or Figure 6 and / or Figure 7 The third portion of the machine-readable instructions represented by the flowchart of can be executed by an ASIC. It should be understood that Figure 2 Some or all of the circuits may therefore be instantiated at the same or different times. Some or all of the circuits may be instantiated, for example, in one or more concurrent and / or serially executed threads. Furthermore, in some examples, Figure 2 Some or all of the circuitry of may be implemented in one or more virtual machines and / or containers executing on a microprocessor.

[0164] In some examples, Fig. 9 The processor circuit 912 may be in one or more packages. For example, Fig.10 The microprocessor 1000 and / or Fig.11 The FPGA circuit 1100 can be in one or more packages. In some examples, the XPU can be composed of Fig. 9 The XPU may be implemented in a processor circuit 912 of the embodiment of the present invention, which may be in one or more packages. For example, the XPU may include a CPU in one package, a DSP in another package, a GPU in yet another package, and an FPGA in yet another package.

[0165] Fig.12 A block diagram illustrating an example software distribution platform 1205 for distributing software (eg, Fig. 9 The example software distribution platform 1205 may be implemented by any computer server, data facility, cloud service, etc. that is capable of storing software and transmitting it to other computing devices. The third party may be a customer of the entity that owns and / or operates the software distribution platform 1205. For example, the entity that owns and / or operates the software distribution platform 1205 may be a software (e.g., Fig. 9The third party may be a consumer, user, retailer, OEM, etc. who purchases and / or licenses the software for use and / or resale and / or sublicense. In the illustrated example, the software distribution platform 1205 includes one or more servers and one or more storage devices. The storage device stores the machine-readable instructions 932, which may correspond to the above-described Figure 6 Machine readable instructions and / or operations 600 and / or Figure 7 The example software distribution platform 1205 may be a server or servers that communicate with the example network 1210, which may correspond to the Internet and / or any one or more of the example networks 104, 926 described above. In some examples, one or more servers respond to a request to transfer software to a requesting party as part of a commercial transaction. Payment for the delivery, sale, and / or license of the software may be processed by one or more servers of the software distribution platform and / or by a third-party payment entity. The server enables purchasers and / or licensors to download machine-readable instructions 932 from the software distribution platform 1205. For example, the server may communicate with the example network 1210. Fig. 9 Software corresponding to the example machine readable instructions 932 may be downloaded to the example processor platform 900, and the example processor platform 900 will execute the machine readable instructions 932 to implement Figure 1 and / or Figure 2 In some examples, one or more servers of the software distribution platform 1205 periodically provide, transmit and / or force software updates (e.g., Fig. 9 Example machine readable instructions 932) to ensure that improvements, patches, updates, etc. are distributed and applied to the software at the end-user device.

[0166] From the foregoing, it will be appreciated that example systems, methods, apparatus, and articles of manufacture have been disclosed that can generate hardware-aware machine learning model architectures for multiple domains without the need for training. The examples disclosed herein greatly simplify and accelerate the way (e.g., by data scientists) build ML models, allowing for the use of general-purpose processor circuits (e.g., Xeon processors) and by end users with little technical expertise. For example, by generating a model from inputs indicating the domain in which the model will operate and the target hardware on which the model will be executed, the examples disclosed herein allow end users with limited technical expertise to develop models. As described above, the examples disclosed herein improve upon general-purpose processor circuits (e.g., ResNet, DLRM, BERT models, etc.) by providing popular models that are more lightweight (e.g., require relatively fewer computational resources compared to other NAS methods), have higher inference throughput, and provide the same or nearly the same metrics (e.g., accuracy) as SOTA models. Xeon processors). The disclosed systems, methods, apparatus, and articles of manufacture improve the efficiency of using computing devices by greatly reducing NAS search time. For example, by utilizing a training-free approach, the disclosed examples do not require iterative training and evaluation of candidate architectures. As a result, the search time is significantly reduced compared to general-purpose NAS methods. Therefore, there is no need to use accelerator circuits (e.g., GPUs) to implement the examples disclosed herein. Instead, in terms of computation, cheaper general-purpose processor circuits (e.g., CPUs) can perform the NAS searches disclosed herein. Therefore, the disclosed systems, methods, apparatus, and articles of manufacture are intended to make one or more improvements to the operation of a machine such as a computer or other electronic and / or mechanical device.

[0167] This article discloses example methods, devices, systems, and articles of manufacture for generating hardware-aware machine learning model architectures for multiple domains without training. Further examples and combinations thereof include the following:

[0168] Example 1 includes an apparatus for generating hardware-aware machine learning (ML) model architectures for multiple domains without training, the apparatus comprising: at least one memory; machine-readable instructions; and processor circuitry for instantiating the machine-readable instructions or executing the machine-readable instructions to: generate multiple candidate architectures for the ML model based on target hardware to be used to execute the ML model and a search space applicable to multiple domains, calculate corresponding comprehensive scores for the multiple candidate architectures, the corresponding comprehensive scores being based at least on corresponding latency scores for the multiple candidate architectures, and select an architecture for the ML model from the multiple candidate architectures, the selected architecture corresponding to a comprehensive score associated with the selected architecture that satisfies a criterion.

[0169] Example 2 includes the apparatus of Example 1, wherein the processor circuit selects a supernet for the ML model based on a domain in which the ML model will operate, the supernet comprising one or more candidate architectures for the ML model, the supernet being selected from at least two supernets corresponding to respective domains of the plurality of domains.

[0170] Example 3 includes the apparatus of Example 2, wherein at least two supernets corresponding to respective domains share a supernet framework associated with the ML model.

[0171] Example 4 includes an apparatus of any one of Examples 1, 2, or 3, wherein, to calculate corresponding composite scores for a plurality of candidate architectures, the processor circuit calculates corresponding products of one or more of an expressiveness score, a complexity score, a diversity score, and a significance score with corresponding hyperparameters for a first candidate architecture, and calculates a first composite score corresponding to the first candidate architecture as the product of: (1) a latency score for the first candidate architecture, and (2) the sum of the corresponding products.

[0172] Example 5 includes the apparatus of Example 4, wherein the corresponding hyperparameter is a binary value, and the processor circuit adjusts the corresponding hyperparameter based on a domain corresponding to the hypernet selected for the ML model.

[0173] Example 6 includes the apparatus of any of Examples 1, 2, 3, 4, or 5, wherein the criterion comprises the composite score being a maximum composite score among corresponding composite scores of a plurality of candidate architectures.

[0174] Example 7 includes the apparatus of any one of Examples 1, 2, 3, 4, 5, or 6, wherein the plurality of fields include a field of computer vision, a field of natural language processing, and a field of recommender systems.

[0175] Example 8 includes a non-transitory machine-readable storage medium comprising instructions that, when executed, cause a processor circuit to perform at least the following: generate multiple candidate architectures for a machine learning (ML) model based on target hardware to be used to execute the ML model and a search space applicable to multiple domains, calculate corresponding comprehensive scores for the multiple candidate architectures, the corresponding comprehensive scores are based at least on corresponding latency scores of the multiple candidate architectures, and select an architecture for the ML model from the multiple candidate architectures for the ML model, the selected architecture corresponding to a comprehensive score associated with the selected architecture that satisfies a criterion.

[0176] Example 9 includes the non-transitory machine-readable storage medium of Example 8, wherein the instructions cause the processor circuit to select a supernet for the ML model based on a domain in which the ML model is to operate, the supernet comprising one or more candidate architectures for the ML model, the supernet being selected from at least two supernets corresponding to respective domains of the plurality of domains.

[0177] Example 10 includes the non-transitory machine-readable storage medium of Example 9, wherein at least two supernets corresponding to respective domains share a supernet framework associated with the ML model.

[0178] Example 11 includes a non-transitory machine-readable storage medium of any one of Examples 8, 9, or 10, wherein, to calculate corresponding composite scores for a plurality of candidate architectures, the instructions cause the processor circuit to calculate corresponding products of one or more of an expressiveness score, a complexity score, a diversity score, and a significance score with corresponding hyperparameters for a first candidate architecture, and calculate a first composite score corresponding to the first candidate architecture as the product of: (1) a latency score for the first candidate architecture, and (2) the sum of the corresponding products.

[0179] Example 12 includes the non-transitory machine-readable storage medium of Example 11, wherein the corresponding hyperparameter is a binary value, and the instructions cause the processor circuit to adjust the corresponding hyperparameter based on a domain corresponding to the hypernet selected for the ML model.

[0180] Example 13 includes the non-transitory machine-readable storage medium of any of Examples 8, 9, 10, 11, or 12, wherein the criteria comprises the composite score being a maximum composite score among corresponding composite scores of the plurality of candidate architectures.

[0181] Example 14 includes the non-transitory machine-readable storage medium of any of Examples 8, 9, 10, 11, 12, or 13, wherein the plurality of fields includes the field of computer vision, the field of natural language processing, and the field of recommender systems.

[0182] Example 15 includes a method for generating hardware-aware machine learning (ML) model architectures for multiple domains without training, the method comprising: generating multiple candidate architectures for the ML model based on target hardware to be used to execute the ML model and a search space applicable to multiple domains by executing instructions using a processor circuit; calculating corresponding comprehensive scores of the multiple candidate architectures by executing instructions using a processor circuit, the corresponding comprehensive scores being based at least on corresponding latency scores of the multiple candidate architectures; and selecting an architecture for the ML model from the multiple candidate architectures of the ML model, the selected architecture corresponding to a comprehensive score associated with the selected architecture that satisfies a criterion.

[0183] Example 16 includes the method of Example 15, further comprising selecting a supernet for the ML model based on the domain in which the ML model will operate, the supernet comprising one or more candidate architectures of the ML model, the supernet being selected from at least two supernets corresponding to respective domains of the plurality of domains.

[0184] Example 17 includes the method of Example 16, wherein at least two supernets corresponding to respective domains share a supernet framework associated with the ML model.

[0185] Example 18 includes the method of any one of Examples 15, 16 or 17, further comprising calculating corresponding comprehensive scores of multiple candidate architectures by performing the following operations: calculating corresponding products of one or more of the expressiveness score, complexity score, diversity score, and significance score with corresponding hyperparameters for a first candidate architecture, and calculating a first comprehensive score corresponding to the first candidate architecture as the product of: (1) a latency score of the first candidate architecture, and (2) the sum of the corresponding products.

[0186] Example 19 includes the method of Example 18, wherein the corresponding hyperparameter is a binary value, and the method further comprises adjusting the corresponding hyperparameter based on a domain corresponding to the hypernet selected for the ML model.

[0187] Example 20 includes the method of any of Examples 15, 16, 17, 18, or 19, wherein the criterion comprises the composite score being a maximum composite score among corresponding composite scores of the plurality of candidate architectures.

[0188] Example 21 includes the method of any one of Examples 15, 16, 17, 18, 19, or 20, wherein the plurality of fields includes the field of computer vision, the field of natural language processing, and the field of recommender systems.

[0189] Example 22 includes an apparatus for generating hardware-aware machine learning (ML) model architectures for multiple domains without training, the apparatus comprising: an interface circuit that receives an input indicating a target hardware on which the ML model will be executed; and a processor circuit comprising one or more of the following: at least one of a central processor unit (CPU), a graphics processor unit (GPU), or a digital signal processor (DSP), at least one of the CPU, GPU, or DSP having a control circuit to control data movement within the processor circuit, arithmetic and logic circuits to perform one or more first operations corresponding to an instruction, and one or more registers to store a first result of the one or more first operations, the instruction being in the apparatus; a field programmable gate array (FPGA), the FPGA comprising a first logic gate circuit, a plurality of configurable A processor circuit includes a first logic gate circuit and a plurality of configurable interconnects, and a storage circuit, wherein the first logic gate circuit and the plurality of configurable interconnects perform one or more second operations, and the storage circuit stores second results of the one or more second operations; or an application-specific integrated circuit (ASIC), including a second logic gate circuit to perform one or more third operations; the processor circuit performs at least one of the first operation, the second operation, or the third operation to instantiate: a search engine circuit to generate a plurality of candidate architectures for an ML model based on target hardware and a search space applicable to a plurality of domains; and a predictor circuit to calculate corresponding comprehensive scores of the plurality of candidate architectures, the corresponding comprehensive scores being based at least on corresponding delay scores of the plurality of candidate architectures, the search engine circuit selecting an architecture for the ML model from the plurality of candidate architectures of the ML model, the selected architecture corresponding to a comprehensive score associated with the selected architecture that satisfies a criterion.

[0190] Example 23 includes the apparatus of Example 22, wherein the input is a first input, the interface circuit receives a second input indicating a domain in which the ML model is to operate, and the processor circuit performs at least one of the first operation, the second operation, or the third operation to instantiate a search engine circuit to select a supernet for the ML model based on the domain, the supernet comprising one or more candidate architectures for the ML model, the supernet being selected from at least two supernets corresponding to respective domains of the plurality of domains.

[0191] Example 24 includes the apparatus of Example 23, wherein at least two supernets corresponding to respective domains share a supernet framework associated with the ML model.

[0192] Example 25 includes an apparatus of any one of Examples 22, 23, or 24, wherein, to calculate corresponding composite scores for a plurality of candidate architectures, the processor circuit performs at least one of a first operation, a second operation, or a third operation to instantiate a predictor circuit to calculate corresponding products of one or more of an expressiveness score, a complexity score, a diversity score, and a significance score with corresponding hyperparameters for a first candidate architecture, and calculates a first composite score corresponding to the first candidate architecture as the product of: (1) a latency score for the first candidate architecture, and (2) a sum of the corresponding products.

[0193] Example 26 includes the apparatus of Example 25, wherein the corresponding hyperparameter is a binary value, and the processor circuit performs at least one of the first operation, the second operation, or the third operation to instantiate the predictor circuit to adjust the corresponding hyperparameter based on the domain corresponding to the supernet selected for the ML model.

[0194] Example 27 includes the apparatus of any of Examples 22, 23, 24, 25, or 26, wherein the criterion includes the composite score being the maximum composite score among the corresponding composite scores.

[0195] Example 28 includes the apparatus of any of Examples 22, 23, 24, 25, 26, or 27, wherein the plurality of fields include a field of computer vision, a field of natural language processing, and a field of recommender systems for the plurality of candidate architectures.

[0196] The following claims are hereby incorporated by reference into the detailed description. Although certain example systems, methods, devices, and articles have been disclosed herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all systems, methods, devices, and articles that fairly fall within the scope of the claims of this patent.

Claims

1. An apparatus for generating hardware-aware machine learning (ML) model architectures for multiple domains without training, the apparatus comprising: at least one memory; machine-readable instructions; as well as Processor circuitry for instantiating the machine-readable instructions or executing the machine-readable instructions to: generating a plurality of candidate architectures for the ML model based on target hardware on which the ML model will be executed and a search space applicable to the plurality of domains; calculating respective composite scores for the plurality of candidate architectures, the respective composite scores being based at least on respective latency scores for the plurality of candidate architectures, and An architecture is selected for the ML model from the plurality of candidate architectures for the ML model, the selected architecture corresponding to a composite score associated with the selected architecture that satisfies a criterion.

2. The device according to claim 1, wherein: The processor circuit is used to select a supernet for the ML model based on the domain in which the ML model will operate, the supernet including one or more candidate architectures for the ML model, the supernet being selected from at least two supernets corresponding to respective domains of the plurality of domains.

3. The device according to claim 2, wherein: The at least two supernets corresponding to the respective domains share a supernet framework associated with the ML model.

4. The device according to claim 1, wherein: To calculate the respective composite scores for the plurality of candidate architectures, the processor circuit is configured to: Calculating, for the first candidate architecture, corresponding products of one or more of an expressiveness score, a complexity score, a diversity score, and a significance score and corresponding hyperparameters; and A first composite score corresponding to the first candidate architecture is calculated as the product of: (1) the latency score of the first candidate architecture, and (2) the sum of the corresponding products.

5. The device according to claim 4, wherein: The respective hyperparameter is a binary value, and the processor circuit is configured to adjust the respective hyperparameter based on a domain corresponding to a hypernet selected for the ML model.

6. The device according to claim 1, wherein: The criterion includes: the comprehensive score being the maximum comprehensive score among the corresponding comprehensive scores of the plurality of candidate architectures.

7. The device according to claim 1, wherein: The multiple fields include computer vision, natural language processing and recommendation system.

8. A non-transitory machine-readable storage medium comprising instructions that, when executed, cause a processor circuit to perform at least the following: generating a plurality of candidate architectures for a machine learning (ML) model based on target hardware on which the ML model will be executed and a search space applicable to multiple domains; calculating respective composite scores for the plurality of candidate architectures, the respective composite scores being based at least on respective latency scores for the plurality of candidate architectures; and An architecture is selected for the ML model from the plurality of candidate architectures for the ML model, the selected architecture corresponding to a composite score associated with the selected architecture that satisfies a criterion.

9. The non-transitory machine-readable storage medium of claim 8, wherein: The instructions cause the processor circuit to select a supernet for the ML model based on a domain in which the ML model will operate, the supernet comprising one or more candidate architectures for the ML model, the supernet being selected from at least two supernets corresponding to respective domains of the plurality of domains.

10. The non-transitory machine-readable storage medium of claim 9, wherein: The at least two supernets corresponding to the respective domains share a supernet framework associated with the ML model.

11. The non-transitory machine-readable storage medium of claim 8, wherein: To calculate the respective composite scores for the plurality of candidate architectures, the instructions cause the processor circuit to: Calculating, for the first candidate architecture, corresponding products of one or more of an expressiveness score, a complexity score, a diversity score, and a significance score and corresponding hyperparameters; and A first composite score corresponding to the first candidate architecture is calculated as the product of: (1) the latency score of the first candidate architecture, and (2) the sum of the corresponding products.

12. The non-transitory machine-readable storage medium of claim 11, wherein: The respective hyperparameter is a binary value, and the instructions cause the processor circuit to adjust the respective hyperparameter based on a domain corresponding to a supernet selected for the ML model.

13. The non-transitory machine-readable storage medium of claim 8, wherein: The criterion includes: the comprehensive score being the maximum comprehensive score among the corresponding comprehensive scores of the plurality of candidate architectures.

14. The non-transitory machine-readable storage medium of claim 8, wherein: The multiple fields include computer vision, natural language processing and recommendation system.

15. A method for generating hardware-aware machine learning (ML) model architectures for multiple domains without training, the method comprising: generating, by executing instructions with a processor circuit, a plurality of candidate architectures for the ML model based on target hardware on which the ML model will be executed and a search space applicable to the plurality of domains; Calculating, by executing instructions with the processor circuit, respective composite scores for the plurality of candidate architectures, the respective composite scores being based at least on respective latency scores for the plurality of candidate architectures; and An architecture is selected for the ML model from the plurality of candidate architectures for the ML model, the selected architecture corresponding to a composite score associated with the selected architecture that satisfies a criterion.

16. The method of claim 15, further comprising selecting a supernet for the ML model based on the domain in which the ML model will operate, the supernet comprising one or more candidate architectures for the ML model, the supernet being selected from at least two supernets corresponding to respective domains of the plurality of domains.

17. The method according to claim 16, wherein: The at least two supernets corresponding to the respective domains share a supernet framework associated with the ML model.

18. The method according to claim 15, further comprising calculating the corresponding comprehensive scores of the plurality of candidate architectures by: Calculating, for the first candidate architecture, corresponding products of one or more of an expressiveness score, a complexity score, a diversity score, and a significance score and corresponding hyperparameters; and A first composite score corresponding to the first candidate architecture is calculated as the product of: (1) the latency score of the first candidate architecture, and (2) the sum of the corresponding products.

19. The method according to claim 18, wherein: The corresponding hyperparameter is a binary value, and the method further includes adjusting the corresponding hyperparameter based on a domain corresponding to a supernet selected for the ML model.

20. The method according to claim 15, wherein: The criterion includes: the comprehensive score being the maximum comprehensive score among the corresponding comprehensive scores of the plurality of candidate architectures.

21. The method according to claim 15, wherein: The multiple fields include computer vision, natural language processing and recommendation system.

22. An apparatus for generating hardware-aware machine learning (ML) model architectures for multiple domains without requiring training, the apparatus comprising: an interface circuit for receiving an input indicative of target hardware on which the ML model will be executed; as well as A processor circuit comprising one or more of the following: at least one of a central processor unit (CPU), a graphics processor unit (GPU), or a digital signal processor (DSP), the at least one of the CPU, the GPU, or the DSP having control circuitry for controlling data movement within the processor circuitry, arithmetic and logic circuitry for performing one or more first operations corresponding to an instruction, and one or more registers for storing a first result of the one or more first operations, the instruction being in the apparatus; a field programmable gate array (FPGA), the FPGA comprising a first logic gate circuit, a plurality of configurable interconnects, and a storage circuit, the first logic gate circuit and the plurality of configurable interconnects being used to perform one or more second operations, the storage circuit being used to store a second result of the one or more second operations; or an application specific integrated circuit (ASIC) comprising a second logic gate circuit for performing one or more third operations; The processor circuit is configured to perform at least one of the first operation, the second operation, or the third operation to instantiate: a search engine circuit for generating a plurality of candidate architectures for the ML model based on the target hardware and a search space applicable to the plurality of domains; as well as A predictor circuit is configured to calculate respective composite scores for the plurality of candidate architectures, the respective composite scores being based at least on respective latency scores for the plurality of candidate architectures, the search engine circuit being configured to select an architecture for the ML model from the plurality of candidate architectures for the ML model, the selected architecture corresponding to a composite score associated with the selected architecture that satisfies a criterion.

23. The apparatus of claim 22, wherein: The input is a first input; The interface circuit is to receive a second input indicating a domain in which the ML model is to operate; and The processor circuit is used to: perform at least one of the first operation, the second operation, or the third operation to instantiate the search engine circuit to select a supernet for the ML model based on the domain, the supernet including one or more candidate architectures of the ML model, the supernet being selected from at least two supernets, the at least two supernets corresponding to respective domains of the plurality of domains.

24. The apparatus of claim 23, wherein the at least two supernets corresponding to the respective domains share a supernet framework associated with the ML model.

25. The device according to claim 22, wherein: To calculate the respective composite scores for the plurality of candidate architectures, the processor circuit is configured to perform at least one of the first operation, the second operation, or the third operation to instantiate the predictor circuit to perform the following operations: Calculating, for the first candidate architecture, corresponding products of one or more of an expressiveness score, a complexity score, a diversity score, and a significance score and corresponding hyperparameters; and A first composite score corresponding to the first candidate architecture is calculated as the product of: (1) the latency score of the first candidate architecture, and (2) the sum of the corresponding products.

26. The apparatus of claim including example 25, wherein: The corresponding hyperparameter is a binary value, and the processor circuit is used to perform at least one of the first operation, the second operation, or the third operation to instantiate the predictor circuit to adjust the corresponding hyperparameter based on a domain corresponding to the supernet selected for the ML model.

27. The device according to claim 22, wherein: The criterion includes: the comprehensive score being the maximum comprehensive score among the corresponding comprehensive scores of the plurality of candidate architectures.

28. The device according to claim 22, wherein: The multiple fields include computer vision, natural language processing and recommendation system.

Images