Methods and systems for the automated generation of neural network architectures

Abstract

Presented herein are systems and methods for generating a neural network architecture (NNA) tailored for a given task. In certain embodiments, the technology automatically identifies a neural network architecture appropriate to perform the task, tweaks the architecture to meet one or more particular use-case requirements, and trains the most optimal neural network model for the task.

Description

Attorney Docket No.2010363-0434 METHODS AND SYSTEMS FOR THE AUTOMATED GENERATION OF NEURAL NETWORK ARCHITECTURES PRIORITY APPLICATION

[0001] This application claims the benefit of U.S. Provisional Patent No.63 / 627,621, filed on January 31, 2024, the disclosure of which is hereby incorporated by reference herein in its entirety. FIELD

[0002] This invention relates generally to systems and methods for the automated generation of artificial neural network architectures. BACKGROUND

[0003] Artificial neural networks are widely used in various domains such as manufacturing, transportation, smart cities, healthcare, preventive medicine, retail, surveillance, banking, and finance. These models are designed to process various types of data, for example, images, videos, sound, sensor values, alphanumeric data, and natural language documents, and produce corresponding output based on sets of training data. Artificial neural networks are becoming increasingly complex, requiring heavy computational energy to search for optimal neural architecture models, train the models, and inference the models post deployment (e.g., use the trained models to produce output, e.g., a response to a query, given a set of input).

[0004] There is a need for systems and methods to automatically and efficiently design neural network architectures that are tailored to perform particular tasks.Attorney Docket No.2010363-0434 SUMMARY

[0005] Presented herein are systems and methods for generating a neural network architecture (NNA) tailored for a given task (e.g., a classification, a regression, or the like). In certain embodiments, the technology automatically identifies a neural network architecture appropriate to perform a given task, modifies the architecture to meet one or more particular use- case requirements, and trains the most optimal neural network model for the given task. In certain embodiments, the technology provides a complete end-to-end solution that does not require a user to have Artificial Intelligence (AI) domain knowledge.

[0006] In one aspect, the present disclosure is directed to a method of automatically generating a neural network architecture (NNA) tailored for a given task (e.g., a classification, a regression, or the like), the method comprising the following steps (a) to (f). At step (a), the method comprises receiving, by a processor of a computing device, user input comprising (i) a task description (i.e., a task type, e.g., a classification, a regression, or the like), (ii) one or more dataset(s) associated with the given task (e.g., a training dataset, a validation dataset, and / or a testing dataset), and (iii) one or more input metrics to guide the automatic NNA generation process. At step (b), the method comprises automatically generating, by the processor (e.g., by a Neural Network Generator Expert System, NNGES, of the processor), a command based at least on the received user input. In some embodiments, the command is based on (i) the user input, (ii) a set of rules (e.g., a Neural Network Generator Rule Book), and, optionally, (iii) one or more previously predicted models (e.g., one or more previously generated candidate NNAs) for the given task and / or its / their corresponding evaluation reports. At step (c), the method comprises transmitting, by the processor, the automatically generated command to a Large Language Model (LLM) and / or an Artificial General Intelligence (AGI) model (an LLM / AGI) (e.g., GPT-4 Turbo,Attorney Docket No.2010363-0434 GPT-4o, Gemini), and receiving from the LLM / AGI, in response to said command, a first generated candidate neural network architecture (a first candidate NNA). At step (d), the method comprises evaluating, by the processor, the first candidate NNA by at least partially training the first candidate NNA and validating the first candidate NNA using a subset of the user input (e.g., using a subset of the training dataset and / or a subset of the validation dataset and / or a subset of the testing dataset), thereby determining values of one or more evaluation based metrics associated with the first candidate NNA. At step (e), the method comprises repeating steps (b) to (d) one or more times to generate, by the processor, a second and / or subsequent candidate NNA(s) and to determine associated evaluation based metrics for each candidate NNA (e.g., performing multiple iterations). At step (f), the method comprises identifying, by the processor, a generated candidate NNA (e.g., a trained NNA) as satisfactory (e.g., optimized) for the given task upon attaining a terminating condition [e.g., upon attaining (i) a user-defined limit on a number of iterations in the candidate NNA-generation process, or (ii) a user-defined limit on amount of time that can be used for the candidate NNA-generation process, or (iii) upon achieving acceptable evaluation based metrics for a generated candidate NNA].

[0007] In some embodiments, step (b) comprises automatically performing, by the processor, a procedure to resolve any conflict in the automatically generated command (e.g., any conflict between instructions in a command dictionary generated by the NNGES) (e.g., using a NNGES Rule Book that comprises one or more of the instructions listed in Table 1).

[0008] In some embodiments, the one or more evaluation based metrics associated with a candidate NNA comprise one or more members selected from the group consisting of: (i) a training accuracy of the candidate NNA, (ii) a validation accuracy of the candidate NNA, (iii) an energy required for evaluating the training set, (iv) an energy required for evaluating theAttorney Docket No.2010363-0434 validation set, (v) a value of frames per second (FPS) of the candidate NNA, and (vi) one or more parameters of the candidate NNA.

[0009] In some embodiments, the one or more input metrics to guide the automatic NNA generation process comprise one or more members selected from the group consisting of: (i) a weight for accuracy (e.g., to weight the training and / or the validation accuracy), (ii) a weight for FPS (e.g., to weight the FPS value), and (iii) a weight for energy (e.g., to weight the energy required for training and / or validation).

[0010] In some embodiments, the one or more input metrics to guide the automatic NNA generation process comprise internal metrics associated with internal settings of the LLM / AGI (e.g., a level of creativity of the LLM / AGI).

[0011] In some embodiments, the one or more input metrics to guide the automatic NNA generation process comprise one or more members selected from the group consisting of: (i) a priority of training accuracy, (ii) a threshold of training accuracy, (iii) a priority of validation accuracy, (iv) a threshold of validation accuracy, (v) a priority of energy required for evaluating the training set, (vi) a threshold of energy required for evaluating the training set, (vii) a priority of energy required for evaluating the validation set, (viii) a threshold of energy required for evaluating the validation set, (ix) a priority of FPS, (x) a threshold of FPS, (xi) a priority of each of one or more parameters of a candidate NNA, (xii) a threshold of each of one or more parameters of a candidate NNA, (xiii) a threshold value (e.g., user-defined) to check overfitting (e.g., 10% or more depending on the task description), (xiv) a threshold value (e.g., user-defined) to check underfitting (e.g., 5% or more depending on the task description).

[0012] In some embodiments, the command generated in step (b) comprises a set of architectural commands. In some embodiments, each architectural command in the set ofAttorney Docket No.2010363-0434 architectural commands comprises: (i) a type of neural network architectural change (e.g., according to Table 1; associated with modifying specific structure of neural network layers); and (ii) a weight (e.g., a numerical value) associated with prioritization of the architectural command.

[0013] In some embodiments, the procedure to resolve any conflict in the automatically generated command comprises setting a lower weight in a pair of conflicting architectural commands as zero.

[0014] In some embodiments, the one or more input metrics to guide the automatic NNA generation process are updated in step (e) based on the one or more evaluation based metrics [e.g., at some (e.g., each, selected) iterations; for each of the some iteration, updates are based on the associated evaluation based metrics; e.g., based at least partially on LeMoNaDe Expert System according to Algorithm 3].

[0015] In some embodiments, the one or more input metrics to guide the automatic NNA generation process comprise the threshold of training accuracy (e.g., and the priority of training accuracy). In some embodiments, if the training accuracy of the candidate NNA does not meet (e.g., is less than) the threshold of training accuracy, weights of one or more architectural commands [e.g., Add Convolution Layer (ACL), Add Dense Layer (ADL), Add More Kernel (AMK), Add Skip Connection (ASC)] are increased (e.g., each, e.g., by the priority of training accuracy).

[0016] In some embodiments, the one or more input metrics to guide the automatic NNA generation process comprise the threshold of validation accuracy (e.g., and the priority of validation accuracy). In some embodiments, if the validation accuracy of the candidate NNA does not meet (e.g., is less than) the threshold of validation accuracy, weights of one or more architectural commands [e.g., Add Convolution Layer (ACL), Add Dense Layer (ADL), AddAttorney Docket No.2010363-0434 More Kernel (AMK), Add Skip Connection (ASC)] are increased (e.g., each, e.g., by the priority of validation accuracy).

[0017] In some embodiments, the one or more input metrics to guide the automatic NNA generation process comprise the threshold of energy required for evaluating the training set (e.g., and the priority of energy required for evaluating the training set). In some embodiments, if the energy required for evaluating the training set satisfies (e.g., is larger than) the threshold of energy required for evaluating the training set, weights of one or more architectural commands [e.g., Add Dropout Layer (AD), Add Weight Initializer (AWI), Reduce Convolutional Layer (RCL), Reduce Number of Kernel (RK), Reduce Skip Connection (RSC)] are increased (e.g., each, e.g., by the priority of energy required for evaluating the training set).

[0018] In some embodiments, the one or more input metrics to guide the automatic NNA generation process comprise the threshold of energy required for evaluating the validation set (e.g., and the priority of energy required for evaluating the validation set). In some embodiments, if the energy required for evaluating the validation set satisfies (e.g., is larger than) the threshold of energy required for evaluating the validation set, weights of one or more architectural commands [e.g., Add Dropout Layer (AD), Reduce Convolutional Layer (RCL), Reduce Number of Kernel (RK), Reduce Dense Layer (RDL)] are increased (e.g., each, e.g., by the priority of energy required for evaluating the validation set).

[0019] In some embodiments, the one or more input metrics to guide the automatic NNA generation process comprise the threshold of FPS (e.g., and the priority of FPS). In some embodiments, if the value of FPS does not meet (e.g., is less than) the threshold of FPS, weights of one or more architectural commands [e.g., Add Dropout Layer (AD), Reduce Number ofAttorney Docket No.2010363-0434 Kernel (RK), Reduce Skip Connection (RSC)] are increased (e.g., each, e.g., by the priority of FPS).

[0020] In some embodiments, the one or more input metrics to guide the automatic NNA generation process comprise the threshold value to check overfitting (e.g., and the priority of training accuracy). In some embodiments, if a difference between the training accuracy and the validation accuracy satisfies (e.g., is larger than) the threshold value to check overfitting, one or more weights of architectural commands [e.g., Add Dense Layer (ADL), Reduce Convolutional Layer (RCL), Reduce Number of Kernel (RK), Reduce Neurons (RN), Add Regularization (AR)] are increased (e.g., each, e.g., by the priority of training accuracy).

[0021] In some embodiments, the one or more input metrics to guide the automatic NNA generation process comprise the threshold value to check underfitting (e.g., and the priority of validation accuracy). In some embodiments, if a difference between the validation accuracy and the training accuracy satisfies (e.g., is larger than) the threshold value to check underfitting, one or more weights of architectural commands [e.g., Reduce Dropout Layer (RD), Add Convolutional Layer (ACL), Add More Kernel (AMK), Add Dense Layer (ADL), Add More Neurons (AMN), Add Skip Connection (ASC), Reduce Regularization (RR)] are increased (e.g., each, e.g., by the priority of validation accuracy).

[0022] In some embodiments, the user input in step (a) comprises a task description. In some embodiments, the task description is associated with image recognition.

[0023] In some embodiments, the image recognition is associated with one or more objects, said one or more objects comprise alphanumeric symbols (e.g., digits, letters), and / or cloth (e.g., shirt, trousers, pullover, dress, coat, sneaker, ankle boot), and / or geographical features (e.g., forests, mountains, valleys, lakes, rivers, seas), and / or features in satellite images (e.g.,Attorney Docket No.2010363-0434 residential areas, water bodies, forests, various land uses and land covers), and / or features of the human body and associated conditions (e.g., using chest X-ray images for pneumonia classification), and / or cellular features (e.g., using microscopic images to distinguish infected and uninfected cells, e.g., for malaria).

[0024] In another aspect, the present disclosure is directed to a method of automatically generating a neural network architecture (NNA) tailored for a given task (e.g., a classification, a regression, or the like), the method comprising the following steps (a) to (c). At step (a), the method comprises receiving, by a processor of a computing device, user input comprising (i) a task description (i.e., a task type, e.g., a classification, a regression, or the like), (ii) one or more dataset(s) associated with the given task (e.g., a training dataset, a validation dataset, and / or a testing dataset), and (iii) one or more input metrics to guide the automatic NNA generation process. At step (b), the method comprises automatically generating, by the processor (e.g., by a Neural Network Generator Expert System, NNGES, of the processor), a command based at least on the received user input. At step (c), the method comprises transmitting, by the processor, the automatically generated command to a Large Language Model (LLM) and / or an Artificial General Intelligence (AGI) model (an LLM / AGI) (e.g., GPT-4 Turbo, GPT-4o, Gemini), and receiving from the LLM / AGI, in response to said command, the NNA.

[0025] In some embodiments, the generated command in step (b) is based on (i) the user input, (ii) a set of rules (e.g., a Neural Network Generator Rule Book), and, optionally, (iii) one or more previously predicted models (e.g., one or more previously generated candidate NNAs) for the given task and / or its / their corresponding evaluation reports.

[0026] In another aspect, the present disclosure is directed to a method of automatically generating a neural network architecture (NNA) tailored for a given task (e.g., a classification, aAttorney Docket No.2010363-0434 regression, or the like), the method comprising the following steps (a) to (f). At step (a), the method comprises receiving, by a processor of a computing device, user input comprising (i) a task description (i.e., a task type, e.g., a classification, a regression, or the like), (ii) one or more dataset(s) associated with the given task (e.g., a training dataset, a validation dataset, and / or a testing dataset), (iii) one or more input metrics to guide the automatic NNA generation process, and, optionally, (iv) an initial NNA. At step (b), the method comprises automatically generating, by the processor (e.g., by a Neural Network Discovery with Beam Search, NNDBS, of the processor), a command based at least on the received user input. In some embodiments, the command is based on (i) the user input and, optionally, (ii) one or more previously predicted models (e.g., one or more previously generated candidate NNAs) for the given task and / or its / their corresponding evaluation reports. At step (c), the method comprises transmitting, by the processor, the automatically generated command to a Large Language Model (LLM) and / or an Artificial General Intelligence (AGI) model (an LLM / AGI) (e.g., GPT-4 Turbo, GPT-4o, Gemini), and receiving from the LLM / AGI, in response to said command, a first generated candidate neural network architecture (a first candidate NNA). At step (d), the method comprises evaluating, by the processor, the first candidate NNA by at least partially training the first candidate NNA and validating the first candidate NNA using a subset of the user input (e.g., using a subset of the training dataset and / or a subset of the validation dataset and / or a subset of the testing dataset), thereby determining values of one or more evaluation based metrics associated with the first candidate NNA. At step (e), the method comprises repeating steps (b) to (d) one or more times to generate, by the processor, a second and / or subsequent candidate NNA(s) and to determine associated one or more evaluation based metrics for each candidate NNA (e.g., performing multiple iterations). At step (f), identifying, by the processor, a generatedAttorney Docket No.2010363-0434 candidate NNA (e.g., a trained NNA) as satisfactory (e.g., optimized) for the given task upon attaining a terminating condition [e.g., upon attaining (i) a user-defined limit on a number of iterations in the candidate NNA-generation process, or (ii) a user-defined limit on amount of time that can be used for the candidate NNA-generation process, or (iii) upon achieving acceptable one or more evaluation based metrics for a generated candidate NNA].

[0027] In some embodiments, a method of automatically generating a neural network architecture (NNA) tailored for a given task (e.g., a classification, a regression, or the like) comprises step (g). At step (g), the method comprises filtering, by the processor, the candidate NNA(s) by: (1) sorting the candidate NNA(s) according to associated one or more evaluation based metrics into a search tree and (2) generating selected candidate NNA(s) using the search tree according to a search algorithm.

[0028] In some embodiments, the search algorithm comprises a beam search with a beam width.

[0029] In some embodiments, a method of automatically generating a neural network architecture (NNA) tailored for a given task (e.g., a classification, a regression, or the like) comprises step (h). At step (h), the method comprises repeating steps (b) to (g) one or more times to generate, by the processor, revised candidate NNA(s) and to determine associated one or more evaluation based metrics for each revised candidate NNA (e.g., performing multiple revised iterations). In some embodiments, for step (h) when repeating step (b), the user input comprises one of the selected candidate NNA(s). In some embodiments, for step (h) when repeating step (b), the one or more input metrics to guide the automatic NNA generation process are updated based on the one or more evaluation based metrics. In some embodiments, for stepAttorney Docket No.2010363-0434 (h) when repeating step (g), resulting generated candidate NNA(s) and associated one or more evaluation based metrics are appended to the search tree.

[0030] In some embodiments, the one of the selected candidate NNA(s) is selected from the search tree in a sequential order.

[0031] In some embodiments, the one of the selected candidate NNA(s) is selected from the search tree using an algorithm (e.g., a neural network).

[0032] In some embodiments, one or more evaluation based metrics comprise an inference accuracy of the candidate NNA and / or an inference time of the candidate NNA.

[0033] In some embodiments, the one or more input metrics to guide the automatic NNA generation process comprise a weight of accuracy and / or a weight of inference time.

[0034] In another aspect, the present disclosure is directed to a system comprising a processor of a computing device and a memory having instructions stored thereon, which, when executed by the processor, cause the processor to perform any of the methods described herein. BRIEF DECRIPTION OF THE DRAWING

[0035] The foregoing and other objects, aspects, features, and advantages of the present disclosure will become more apparent and better understood by referring to the following description taken in conjunction with the accompanying drawings, in which:

[0036] FIG.1 is a block flow diagram of a method for generating a neural network architecture tailored for a given task, according to an illustrative embodiment.

[0037] FIG.2 is a block diagram of a method for generating a neural network architecture tailored for a given task, called LeMo-NADe, according to an illustrative embodiment.Attorney Docket No.2010363-0434

[0038] FIG.3 is a block flow diagram of a method for generating a neural network architecture tailored for a given task, according to an illustrative embodiment.

[0039] FIG.4A is a block diagram of a method for generating a neural network architecture tailored for a given task, called Beam Search, according to an illustrative embodiment.

[0040] FIG.4B is a block diagram of a method for generating a neural network architecture tailored for a given task, called Beam Search, according to an illustrative embodiment.

[0041] FIG.5 is a block diagram of an exemplary cloud computing environment, used in certain embodiments.

[0042] FIG.6 is a block diagram of an example computing device and an example mobile computing device used in certain embodiments.

[0043] FIG.7 is a block diagram of a method for generating a neural network architecture tailored for a given task, according to an illustrative embodiment.

[0044] FIG.8A is a block diagram of a neural network architecture generated for an image recognition task in the CIFAR-10 dataset with a validation accuracy priority of 1.0, according to an illustrative embodiment.

[0045] FIG.8B is a block diagram of a neural network architecture generated for an image recognition task in the CIFAR-10 dataset with a training energy priority of 0.5 and a validation energy priority of 0.5, according to an illustrative embodiment.

[0046] FIG.9A is a plot of validation and training accuracies for various models operating to perform dataset-specific tasks, according to an illustrative embodiment.Attorney Docket No.2010363-0434

[0047] FIG.9B is a plot of training and validation loss functions for various models operating to perform dataset-specific tasks, according to an illustrative embodiment.

[0048] FIG.10A is a block diagram of various neural network architectures generated by LeMo-NADe with GPT-4o, according to an illustrative embodiment.

[0049] FIG.10B is a block diagram of various neural network architectures generated by LeMo-NADe with GPT-4 Turbo, according to an illustrative embodiment.

[0050] FIG.10C is a block diagram of various neural network architectures generated by LeMo-NADe with Gemini, according to an illustrative embodiment.

[0051] FIG.11 is a block diagram of a neural network architecture generated by LeMo- NADe with GPT-4 Turbo for the CIFAR-10 dataset with temperature 0.6 and setting 3 from Table 3, providing 90.90% test accuracy, according to an illustrative embodiment.

[0052] FIG.12 is a block diagram of a neural network architecture generated by LeMo- NADe with GPT-4 Turbo for the CIFAR-100 dataset with temperature 0.4 and setting 1 from Table 3, providing 57.74% test accuracy, according to an illustrative embodiment.

[0053] FIG.13 is a block diagram of a neural network architecture generated by LeMo- NADe with GPT-4 Turbo for the ImageNet16-120 dataset with temperature 0.4 and setting 1 from Table 3, providing 25.97% test accuracy, according to an illustrative embodiment.

[0054] FIG.14A is a bar chart of accuracy values for different datasets and neural network architecture variants produced by Beam Search (either with 1.0 weight of accuracy or with 0.5 weight of accuracy and 0.5 weight on inference time) with a beam width of 3, according to an illustrative embodiment.

[0055] FIG.14B is a bar chart of accuracy values for different datasets and neural network architecture variants produced by Beam Search (either with 1.0 weight of accuracy orAttorney Docket No.2010363-0434 with 0.5 weight of accuracy and 0.5 weight on inference time) with a beam width of 5, according to an illustrative embodiment.

[0056] FIG.14C is a bar chart of inference time values for different datasets and neural network architecture variants produced by Beam Search (either with 1.0 weight of accuracy or with 0.5 weight of accuracy and 0.5 weight on inference time) with a beam width of 3, according to an illustrative embodiment.

[0057] FIG.14D is a bar chart of inference time values for different datasets and neural network architecture variants produced by Beam Search (either with 1.0 weight of accuracy or with 0.5 weight of accuracy and 0.5 weight on inference time) with a beam width of 5, according to an illustrative embodiment.

[0058] The features and advantages of the present disclosure will become more apparent from the detailed description set forth below when taken in conjunction with the drawings, in which like reference characters identify corresponding elements throughout. In the drawings, like reference numbers generally indicate identical, functionally similar, and / or structurally similar elements. DETAILED DESCRIPTION

[0059] Technologies of the present disclosure, among other things, generate and / or optimize neural network architectures tailored to a given task. A neural network is said to be satisfactory to perform (e.g., is tailored to) a given task once a certain performance of the neural network is reached (e.g., demonstrated). The performance may be associated with various parameters, such as an accuracy of the neural network when performing a specific task (e.g., evaluating a training set as compared to a ground truth, evaluating a validation set as comparedAttorney Docket No.2010363-0434 to a ground truth), energy required to perform a specific task (e.g., evaluate a training set, evaluate a validation set), a maximum number of frames per second (FPS) that a neural network can intake to evaluate each frame in real time, and the like.

[0060] A neural network architecture determines the performance of a neural network. The architecture is associated with a vast number of parameters, such as type, order, size, and number of layers of a neural network, specific connections, and topology in between the layers, associated hyperparameters of the layers and the network.

[0061] Some neural architecture searches (NAS) are subject to the limitations of human knowledge and understanding of underlying principles of artificial intelligence. Other neural architecture searches rely on training neural networks that incorporate a very large number of possible architectures within a search domain. Examples include SuperNet and Differentiable Architecture Search (DARTS). However, such large networks require high computational power and extensive running times. Furthermore, these networks are difficult to reproduce and challenging to implement.

[0062] Yet another NAS approach relies on specific search space (e.g., a limited list of neural network architectures, a limited list of neural network layer types) to identify the best architecture. For example, researchers proposed NAS benchmark dataset that contains all the possible architectures with their corresponding evaluation results. One NAS dataset is NAS- Bench-101, which contains 5 million distinct neural architectures and is trained on CIFAR-10 dataset. The NAS-Bench-201 dataset has 15625 cell layouts and is derived from a cell-based search technique and is trained on CIFAR-10, CIFAR-100 and ImageNet16-120 datasets. NAS method named β-DARTS was proposed to solve the weak generalization ability found in the DARTS method, relying on the NAS-Bench-201 for evaluation. The Λ-DARTS was proposed asAttorney Docket No.2010363-0434 a solution for the structural flaws caused by the weight-sharing approach in DARTS. However, such searches are constrained by the pre-defined search space and are difficult to scale across different applications and use cases.

[0063] Recently, GPT-4 Enhanced Neural archItectUre Search (GENIUS) was proposed to use an LLM to solve the NAS problem. GENIUS, while focused on accuracy, has not considered energy efficiency and environmental parameters for neural network design and use.

[0064] Any two or more of the features described in this specification may be combined to form implementations of the disclosure, whether specifically expressly described as a separate combination in this specification or not.

[0065] It is contemplated that systems, architectures, devices, methods, and processes of the claimed invention encompass variations and adaptations developed using information from the embodiments described herein. Adaptation and / or modification of the systems, architectures, devices, methods, and processes described herein may be performed, as contemplated by this description.

[0066] Throughout the description, where articles, devices, systems, and architectures are described as having, including, or comprising specific components, or where processes and methods are described as having, including, or comprising specific steps, it is contemplated that, additionally, there are articles, devices, systems, and architectures of the present invention that consist essentially of, or consist of, the recited components, and that there are processes and methods according to the present invention that consist essentially of, or consist of, the recited processing steps.Attorney Docket No.2010363-0434

[0067] It should be understood that the order of steps or order for performing certain action is immaterial so long as the invention remains operable. Moreover, two or more steps or actions may be conducted simultaneously.

[0068] The mention herein of any publication, for example, in the Background or the References section, is not an admission that the publication serves as prior art with respect to any of the claims presented herein. The Background section and the References section are presented for purposes of clarity and are not meant as a description of prior art with respect to any claim.

[0069] Documents are incorporated herein by reference as noted. Where there is any discrepancy in the meaning of a particular term, the meaning provided in the Definition section above is controlling.

[0070] Headers are provided for the convenience of the reader – the presence and / or placement of a header is not intended to limit the scope of the subject matter described herein. A. Neural Network Generator Expert System

[0071] In certain embodiments, a method for generating a neural network architecture tailored for a given task is based at least in part on a Large Language Model (LLM) and / or an Artificial General Intelligence (AGI) model (an LLM / AGI) (e.g., GPT-4 Turbo, GPT-4o, Gemini). The exemplary method 100 is shown in FIG.1. Specifically, at step 102, a user input is received. The user input may comprise (i) a task description, (ii) associated datasets, and (iii) generation metrics. The task description, i.e., a task type, may be a classification or a regression. The associated datasets may comprise a training set, a validation set, a testing set. At step 104, a command is generated based on the user input. The command may be based on (i) the user input, (ii) a set of rules, and / or (iii) previously predicted models for the given task. At step 106,Attorney Docket No.2010363-0434 the command is transmitted to an LLM / AGI and a neural network architecture is received in response from the LLM / AGI. The command may use conversational language, for example: “Please suggest an architecture based on the following settings…” or “This architecture achieves XX %, please suggest a better one….”

[0072] Steps 108-112 are optional. At step 108, the NNA is evaluated to determine values of evaluation metrics for the NNA. The evaluation may be associated based at least using a subset of the training dataset and / or a subset of the validation dataset and / or a subset of the testing dataset. At step 110, steps 104-108 are repeated one or many times to generate NNA(s) and associated evaluation metrics (e.g., performing multiple iterations). A number of times that steps 104-108 are repeated may be user-defined. A number of times that steps 104-108 are repeated may be determined based on the evaluation metrics. A number of times that steps 104- 108 are repeated may be determined based on a conditional basis. At step 112, a generated NNA is identified as satisfactory (e.g., optimized) upon satisfying a terminating condition. The termination condition may comprise (i) a user-defined limit on a number of iterations in the NNA-generation process, or (ii) a user-defined limit on amount of time that can be used for the NNA-generation process, or (iii) upon achieving acceptable evaluation based metrics for a generated NNA.

[0073] The LLM / AGI may produce search-space agnostic neural network architectures. The LLM / AGI-produced neural network architectures may demonstrate superior performance (e.g., accuracy, energy budget, FPS) as compared to other neural network architectures.

[0074] In certain embodiments, a user input received in step (a) comprises internal metrics associated with internal settings of the LLM / AGI. The internal metrics may comprise a level of creativity of the LLM / AGI.Attorney Docket No.2010363-0434 i. Architectural Commands and Conflict Resolution

[0075] In certain embodiments, a command generated in step 104 comprises a set of architectural commands. Each architectural command in the set of architectural commands may comprise: (i) a type of neural network architectural change; and (ii) a weight (e.g., a numerical value) associated with prioritization of the architectural command. An exemplary list of the neural network architectural changes is provided in Table 1. A type of neural network architectural change may be associated with modifying specific structure of neural network layers. A higher weight value may be associated with prioritization of associated architectural command over another architectural command associated a lower weight value. A weight value of zero may be associated with forbiddance of associated architectural command.

[0076] In certain embodiments, step 104 comprises a procedure to resolve any conflict in a command. For example, any conflict may comprise any conflict between instructions in a set of architectural commands. Table 1 shows an exemplary Rule Book that the procedure to resolve any conflict may use. Table 1. Rule Book for conflict resolution. LEGEND DESCRIPTION CONFLICTING INSTRUCTIONS ACL ADD CONVOLUTIONAL LAYER RCL ASC ADD SKIP CONNECTION RSC ADL ADD DENSE LAYER RDL RCL REDUCE CONVOLUTIONAL LAYER ACL RSC REDUCE SKIP CONNECTION ASC RDL REDUCE DENSE LAYER ADL AD ADD DROPOUT LAYER RD AMK ADD MORE KERNEL RK AWI ADD WEIGHT INITIALIZER - AR ADD REGULARIZATION RR RK REDUCE NUMBER OF KERNEL AMK RD REDUCE DROPOUT LAYER ADAttorney Docket No.2010363-0434 AMN ADD MORE NEURONS RN RN REDUCE NEURONS AMN

[0077] comprises setting a lower weight in a pair of conflicting architectural commands as zero. Algorithm 1 shows an exemplary algorithm for conflict resolution with cmd referring to the command generated in step 104 with multiple entries each corresponding to a type of neural network architectural change and a value of each entry corresponding to a weight value. A “conflict” property for a given entry identifies a conflicting entry. For example, ASC - Add Skip Connection conflicts with RSC - Remove Skip Connection. In line 3, entries of the command are organized in descending order according to associated values. Next, the cmd is sequentially called starting from an entry with the highest value (weight) and going gradually downward (line 6). Through lines 7-9, command entries with values that are larger than zero are identified and stored in an output value Refined_Cmd. In lines 10-15, the algorithm examines the command entries in pairs to identify any conflicts. If a conflict is detected, a lower value in an associated pair of entries is set to zero. This process eliminates any conflict rules / instructions inside the cmd while prioritizing rules / instructions with higher value(s) / weight(s). The final Refined_Cmd is returned in line 19. Algorithm 1. Conflict Resolution. 1: function RESOLVE_ CONFLICT(cmd) 2: Re f ined_Cmd ← Ø 3: sort_descending (cmd) 4: m ← 0 5: key ← cmd.keys() 6: while m < Length(cmd) doAttorney Docket No.2010363-0434 7: if cmd [key[m]] > 0 then 8: Re f ined _Cmd [key[m]] ← cmd[key[m]] 9: n ← m + 1] then ← 13: end 14: n ← 15: end16: end if 17: m ← m + 1 18: end while 19: return Re f ined_Cmd 20: end function ii. Evaluation Metrics

[0078] In certain embodiments, a user input received in step 102 comprises: (i) a weight for accuracy (e.g., to weight the training and / or the validation accuracy), and / or (ii) a weight for FPS (e.g., to weight the FPS value), and / or (iii) a weight for energy (e.g., to weight the energy required for training and / or validation).

[0079] In certain embodiments, the evaluation metrics associated with an NNA comprise: (i) a training accuracy of the candidate NNA, and / or (ii) a validation accuracy of the candidate NNA, and / or (iii) an energy required for evaluating the training set, and / or (iv) an energy required for evaluating the validation set, and / or (v) a value of frames per second (FPS) of the candidate NNA, and / or (vi) one or more parameters of the candidate NNA. Evaluation based metrics may be associated with a combined model effectiveness. The combined model effectiveness is defined as: CM = AW × (Ta+Va) + FW × NF – EW × (TNE+VNE),Attorney Docket No.2010363-0434 where AW is the weight for accuracy, Tais the training accuracy, Vais the validation accuracy, FW is the weight for FPS, NF is a normalized FPS value, EW is the weight for energy, TNEis the normalized training energy, and VNEis the normalized validation energy. iii. Iterations and Feedback

[0080] In certain embodiments, the generation metrics from a user input are updated based at least in part on evaluation metrics associated with an NNA. The updates may be performed at some (e.g., each, selected) iteration steps. The updates result in a feedback system for neural network architecture generation, where the user input changes at least in part based on evaluation metrics associated with an NNA. Such feedback system may demonstrate superior performance (e.g., accuracy, energy budget) as compared to other neural network architectures (e.g., without a feedback system).

[0081] FIG.2 shows an exemplary implementation of a method for generating a neural network architecture tailored for a given task, called Neural Network Generator Expert System (NNGES). The NNGES takes a user input in the form of a user task specification and generates a command for an LLM using a set of rules for creating a neural architecture from a Rule Book. The NNGES may prioritize user specified metrics from the user input to generate the command. The generated neural network from the LLM is then evaluated and the associated evaluation metrics are used by NNGES to generate a next LLM command.

[0082] Algorithm 2 shows an exemplary algorithm for generating a neural architecture. A user input includes user-defined metrics (UDM), task specification (TS), termination condition (TC), and the initial command (Initial_Cmd). In lines 2-6, all variables are initialized and the initial command (designed based on the task specification) is stored in the command cmd variable. The neural network architecture (NNA) generation process loop is shown in lines 7-25.Attorney Docket No.2010363-0434 In line 8, an LLM_AGI returns a response in the form of an NNA based on the command. The neural network architecture (NNA) model description inside the response is used to create a neural network model Model (line 9). The total number of parameters in Model is extracted and stored inside Metrics.P (line 10). From lines 11-13, the NNA is trained and evaluated with the training and validation datasets, respectively, and a set of metrics, such as training energy (E1), a training accuracy (A1), validation energy (E2) and validation accuracy (A2), are calculated and stored in the Metrics variable. The Metrics is then passed to the NNGES (e.g., using Algorithm 3) to generate a set of instructions for the next round of LLM / AGI-based neural network generation. Any conflicts between the generated instructions may be removed using Algorithm 1. The set of instructions is used in the next iteration of the neural network architecture generation process (if the process is not terminated). In line 16, a single value is computed (CM) that captures the effectiveness of the neural network model in this current iteration. CM is used to keep track of the best neural network model across all iterations of the generation process (lines 17-20). If the terminating condition is met, then the loop is terminated (line 21 and line 22). The best model discovered is returned at the end of the algorithm (line 25). Algorithm 2. Neural Network Generation. 1: function AI_AG (U DM, TS, TC, Initial_Cmd) 2: BCM ← Ø 3: Metrics ← U DM 4: BestModel ← Ø 5: cmd ← Initial_Cmd 6: TC = False 7: while TC = False do 8: Response ← LLM_AGI(cmd) 9: Model ← Create_Model(Response) 10: Metrics.P ← Model.params 11: [Model, Metrics.E1] ← Model.train(TS.train_set)Attorney Docket No.2010363-0434 12: Metrics.A1← Model.eval(train_set) 13: [Metrics.E2,Metrics.A2] ← Model.eval(val) 14: cmd← NNGES(Metrics) 15: cmd← RESOLVE_CONFLICT(cmd) 16: CM ← Model_Effectiveness(Metrics) 17: if CM > BCM then 18: BCM ← CM 19: BestModel ← Model 20: end if 21: if Termination Condition is Met then 22: TC = True 23: end if 24: end while 25: return BestModel 26: end function

[0083] In certain embodiments, a user input received in step 102 comprises: (i) a priority of training accuracy, and / or (ii) a threshold of training accuracy, and / or (iii) a priority of validation accuracy, and / or (iv) a threshold of validation accuracy, and / or (v) a priority of energy required for evaluating the training set, and / or (vi) a threshold of energy required for evaluating the training set, and / or (vii) a priority of energy required for evaluating the validation set, and / or (viii) a threshold of energy required for evaluating the validation set, and / or (ix) a priority of FPS for the model, and / or (x) a threshold of FPS for the model, and / or (xi) a priority of each of one or more parameters of a candidate NNA, and / or (xii) a threshold of each of one or more parameters of a candidate NNA, and / or (xiii) a threshold value (e.g., user-defined) to check overfitting (e.g., 10% or more depending on the task description), and / or (xiv) a threshold value (e.g., user- defined) to check underfitting (e.g., 5% or more depending on the task description). Table 2 shows an exemplary summary of various metrics, where “evaluation based” refers to evaluation metrics and “user defined” refers to the user input.Attorney Docket No.2010363-0434 Table 2. Various metrics used in NNA generation. LEGENDS DESCRIPTION TYPE A1TRAINING ACCURACY OF THE CURRENT MODEL (CM) EVALUATION PREDICTED BY LLM / AGI BASED PA1 PRIORITY OF THE TRAINING ACCURACY USER DEFINED TA1 THRESHOLD OF THE TRAINING ACCURACY USER DEFINED A2 VALIDATION ACCURACY OF THE CURRENT MODEL (CM) EVALUATION PREDICTED BY LLM / AGI BASED PA2 PRIORITY OF THE VALIDATION ACCURACY USER DEFINED TA2 THRESHOLD OF THE VALIDATION ACCURACY USER DEFINED E1 ENERGY REQUIRED FOR EVALUATING THE TRAINING SET EVALUATION BASED PE1PRIORITY OF THE ENERGY REQUIRED FOR EVALUATING THE USER DEFINED TRAINING SET TE1THRESHOLD OF ENERGY REQUIRED FOR EVALUATING THE USER DEFINED TRAINING SET E2ENERGY REQUIRED FOR EVALUATING THE VALIDATION SET EVALUATION BASED PE2 PRIORITY OF THE ENERGY REQUIRED FOR EVALUATING THE USER DEFINED VALIDATION SET TE2 THRESHOLD OF ENERGY REQUIRED FOR EVALUATING THE USER DEFINED VALIDATION SET F FPS OF THE CURRENT MODEL (CM) PREDICTED BY LLM / AGI EVALUATION BASED PF PRIORITY OF THE FPS FOR THE MODEL USER DEFINED TF THRESHOLD OF THE FPS FOR THE MODEL USER DEFINED P PARAMETERS OF THE CURRENT MODEL (CM) PREDICTED BY EVALUATION LLM / AGI BASED OT THRESHOLD VALUE TO CHECK THE OVERFITTING USER DEFINED UT THRESHOLD VALUE TO CHECK THE UNDERFITTING USER DEFINED

[0084] In certain embodiments, a user input comprises a threshold of training accuracy and, if a training accuracy of an NNA does not meet (e.g., is less than) the threshold of training accuracy, weights of one or more architectural commands are increased. The user input may comprise the priority of training accuracy. At least some of the weights may increase by theAttorney Docket No.2010363-0434 priority of training accuracy. The one or more architectural commands may comprise one or more member selected from the group consisting of: Add Convolution Layer (ACL), Add Dense Layer (ADL), Add More Kernel (AMK), and Add Skip Connection (ASC). A respective weight may be communicated in a command as: “A weight of 0.5 is provided to Add Convolution Layer instruction ….”

[0085] In certain embodiments, a user input comprises a threshold of validation accuracy and, if a validation accuracy of an NNA does not meet (e.g., is less than) the threshold of validation accuracy, weights of one or more architectural commands are increased. The user input may comprise the priority of validation accuracy. At least some of the weights may increase by the priority of validation accuracy. The one or more architectural commands may comprise one or more member selected from the group consisting of: Add Convolution Layer (ACL), Add Dense Layer (ADL), Add More Kernel (AMK), and Add Skip Connection (ASC).

[0086] In certain embodiments, a user input comprises a threshold of energy required for evaluating a training set and, if an energy required for evaluating the training set of an NNA satisfies (e.g., is larger than) the threshold of energy required for evaluating the training set, weights of one or more architectural commands are increased. The user input may comprise the priority of energy required for evaluating the training set. At least some of the weights may increase by the priority of energy required for evaluating the training set. The one or more architectural commands may comprise one or more member selected from the group consisting of: Add Dropout Layer (AD), Add Weight Initializer (AWI), Reduce Convolutional Layer (RCL), Reduce Number of Kernel (RK), and Reduce Skip Connection (RSC).

[0087] In certain embodiments, a user input comprises a threshold of energy required for evaluating a validation set and, if an energy required for evaluating the validation set of an NNAAttorney Docket No.2010363-0434 satisfies (e.g., is larger than) the threshold of energy required for evaluating the validation set, weights of one or more architectural commands are increased. The user input may comprise the priority of energy required for evaluating the validation set. At least some of the weights may increase by the priority of energy required for evaluating the validation set. The one or more architectural commands may comprise one or more member selected from the group consisting of: Add Dropout Layer (AD), Reduce Convolutional Layer (RCL), Reduce Number of Kernel (RK), and Reduce Dense Layer (RDL).

[0088] In certain embodiments, a user input comprises a threshold of FPS and, if a value of FPS of an NNA does not meet (e.g., is less than) the threshold of FPS, weights of one or more architectural commands are increased. The user input may comprise the priority of FPS. At least some of the weights may increase by the priority of FPS. The one or more architectural commands may comprise one or more member selected from the group consisting of: Add Dropout Layer (AD), Reduce Number of Kernel (RK), and Reduce Skip Connection (RSC).

[0089] In certain embodiments, the user input comprises a threshold value to check overfitting and, if a difference between a training accuracy and a validation accuracy satisfies (e.g., is larger than) the threshold value to check overfitting, weights of one or more architectural commands are increased. The user input may comprise the priority of training accuracy. At least some of the weights may increase by the priority of training accuracy. The one or more architectural commands may comprise one or more member selected from the group consisting of: Add Dense Layer (ADL), Reduce Convolutional Layer (RCL), Reduce Number of Kernel (RK), Reduce Neurons (RN), and Add Regularization (AR).

[0090] In certain embodiments, a user input comprises a threshold value to check underfitting and, if a difference between a validation accuracy and a training accuracy satisfiesAttorney Docket No.2010363-0434 (e.g., is larger than) the threshold value to check underfitting, weights of one or more architectural commands are increased. The user input may comprise the priority of validation accuracy. At least some of the weights may increase by the priority of validation accuracy. The one or more architectural commands may comprise one or more member selected from the group consisting of: Reduce Dropout Layer (RD), Add Convolutional Layer (ACL), Add More Kernel (AMK), Add Dense Layer (ADL), Add More Neurons (AMN), Add Skip Connection (ASC), and Reduce Regularization (RR).

[0091] Algorithm 3 shows an exemplary algorithm to update the generation metrics. This algorithm is used to analyze the generation metrics Metrics and to propose a command for an LLM / AGI for subsequent iteration(s) for generating a neural network architecture. In line 2, a command cmd containing multiple entries is initialized. Each entry corresponds to a type of neural network architectural change and a value of each entry corresponds to a weight value. Next (lines 3-48), different evaluation-based metrics and user-defined metrics are compared to determine values of each entry. For example, if Metrics.A1 is less than Metrics.TA1, then a value Metrics.PA1 is added to entries associated with instructions ‘ACL, ‘ADL’, ‘AMK, and ‘ASC’. Algorithm 3. NNGES. 1: function NNGES(Metrics) 2: cmd ← 0, AR : 0, RK : 3: if4: cmd[ACL] ← cmd[ACL] + Metrics.PA1 5: cmd[ADL] ← cmd[ADL] + Metrics.PA1 6: cmd[AMK] ← cmd[AMK] + Metrics.PA1 7: cmd[ASC] ← cmd[ASC] + Metrics.PA1 8: end if 9: if Metrics.A2 < Metrics.T A2 then 10: cmd[ACL] ← cmd[ACL] + Metrics.PA2Attorney Docket No.2010363-0434 11: cmd[ADL] ← cmd[ADL] + Metrics.PA212: cmd[AMK] ← cmd[AMK] + Metrics.PA217: cmd[AW I] ← cmd[AW I] + Metrics.PE1 18: cmd[RCL] ← cmd[RCL] + Metrics.PE1 19: cmd[RK] ← cmd[RK] + Metrics.PE1 20: cmd[RSC] ← cmd[RSC] + Metrics.PE1 21: end if 22: if Metrics.E2 < Metrics.T E2 then 23: cmd[AD] ← cmd[AD] + Metrics.PE2 24: cmd[RCL] ← cmd[RCL] + Metrics.PE2 25: cmd[RK] ← cmd[RK] + Metrics.PE226: cmd[RDL] ← cmd[RDL] + Metrics.PE227: end if 28: if Metrics.F < Metrics.T F then 29: cmd[AD] ← cmd[AD] + Metrics.PF 30: cmd[RK] ← cmd[RK] + Metrics.PF 31: cmd[RSC] ← cmd[RSC] + Metrics.PF 32: end if 33: if (Metrics.A1 - Metrics.A2) > Metrics.OT then 34: cmd[ADL] ← cmd[ADL] + Metrics.PA1 35: cmd[RCL] ← cmd[RCL] + Metrics.PA1 36: cmd[RK] ← cmd[RK] + Metrics.PA1 37: cmd[RN] ← cmd[RN] + Metrics.PA1 38: cmd[AR] ← cmd[AR] + Metrics.PA139: end if 40: if (Metrics.A2- Metrics.A1) > Metrics.UT then 41: cmd[RD] ← cmd[RD] + Metrics.PA242: cmd[ACL] ← cmd[ACL] + Metrics.PA243: cmd[AM K] ← cmd[AM K] + Metrics.PA244: cmd[ADL] ← cmd[ADL] + Metrics.PA245: cmd[AM N] ← cmd[AM N] + Metrics.PA246: cmd[ASC] ← cmd[ASC] + Metrics.PA247: cmd[RR] ← cmd[RR] + Metrics.PA2 48: end if 49: return cmd 50: end function B. Beam SearchAttorney Docket No.2010363-0434

[0092] In certain embodiments, a method for generating a neural network architecture tailored for a given task is based at least in part on a Large Language Model (LLM) and / or an Artificial General Intelligence (AGI) model (an LLM / AGI) (e.g., GPT-4 Turbo, GPT-4o, Gemini). The exemplary method 300 is shown in FIG.3. At step 302, a user input is received. The user input may comprise (i) a task description, (ii) associated datasets, and (iii) generation metrics. The task description, i.e., a task type, may be a classification or a regression. The associated datasets may comprise a training set, a validation set, a testing set. At step 304, a command is generated based on the user input. The command may be based on (i) the user input and, optionally, (ii) one or more previously predicted models (e.g., one or more previously generated candidate NNAs) for the given task and / or its / their corresponding evaluation reports. At step 306, the command is transmitted to an LLM / AGI and a neural network architecture is received in response from the LLM / AGI.

[0093] Steps 308-314 are optional. At step 308, the NNA is evaluated to determine values of evaluation metrics for the NNA. The evaluation may be associated based at least using a subset of the training dataset and / or a subset of the validation dataset and / or a subset of the testing dataset. At step 310, steps 304-308 are repeated one or many times to generate NNA(s) and associated evaluation metrics. A number of times that steps 304-308 are repeated may be user-defined. A number of times that steps 304-308 are repeated may be determined based on the evaluation metrics. A number of times that steps 304-308 are repeated may be determined based on a conditional basis. At step 312, a generated NNA is identified as satisfactory (e.g., optimized) upon satisfying a terminating condition. The termination condition may comprise (i) a user-defined limit on a number of iterations in the NNA-generation process, or (ii) a user-Attorney Docket No.2010363-0434 defined limit on amount of time that can be used for the NNA-generation process, or (iii) upon achieving acceptable evaluation based metrics for a generated NNA.

[0094] At step 314, the generated NNA(s) are filtered using an algorithmic search. The generated NNA(s) may be filtered by: (1) sorting the NNA(s) according to associated evaluation metrics into a search tree and (2) generating selected NNA(s) using the search tree according to a search algorithm. The search algorithm may be a heuristic search algorithm. The search algorithm may be a beam search associated with a beam width.

[0095] In certain embodiments, the method comprises a step of repeating steps 304-314 one or more times. For example, steps 304-314 may be repeated one or more times to generate, at each iteration, selected NNA(s) and associated evaluation metrics (e.g., performing multiple revised iterations). A number of times that steps 304-314 are repeated may be user-defined. A number of times that steps 304-314 are repeated may be determined based on the evaluation metrics. A number of times that steps 304-314 are repeated may be determined based on a conditional basis. At every revised iteration, at step 304, the user input may comprise one of the selected NNA(s). The one of the selected NNA(s) may be selected from the search tree in a sequential manner (e.g., based on a value of associated evaluation metrics). The one of the selected NNA(s) may be selected from the search tree using an algorithm (e.g., a neural network trained for the given task). At every revised iteration, at step 304, the generation metrics may be updated based on the evaluation metrics (e.g., associated with a previous revised iteration). At every revised iteration, at step 314, resulting generated NNA(s) and associated evaluation metrics may be appended to the search tree (e.g., before sorting the search tree).

[0096] In certain embodiments, the evaluation metrics comprise an inference accuracy of the candidate NNA and / or an inference time of the candidate NNA. In certain embodiments, theAttorney Docket No.2010363-0434 generation metrics comprise a weight of accuracy and / or a weight of inference time. The evaluation metrics may comprise combine model effectiveness (CM). The combine model effectiveness (CM) considers both the validation accuracy and the inference times, penalizing higher inference times. It may be defined as: CM = Wa× Acc – λ × Wt× N_IT, where Acc is a validation accuracy of a candidate over a validation set, Wais a weight of accuracy, λ is a constant fixed via trial and error, Wtis a weight of inference time, and N_IT is a normalized inference time. The normalized inference time (N_IT) may be calculated as: IT − ITN_IT = ^^^^^^^^^^^^IT, ^^^^^^^^^^^^ − IT^^^^^^^^^^^^where IT is an inference time, ITmintime across various neural architecture searches for a given task on a given dataset, and ITmaxis a maximum inference time across neural architecture searches for a given task on a given dataset.

[0097] In certain embodiments, the generation metrics comprise: (i) a weight for accuracy (e.g., to weight the training and / or the validation accuracy), and / or (ii) a weight for FPS (e.g., to weight the FPS value), and / or (iii) a weight for energy (e.g., to weight the energy required for training and / or validation).

[0098] In certain embodiments, the evaluation metrics comprise: (i) a training accuracy of the candidate NNA, and / or (ii) a validation accuracy of the candidate NNA, and / or (iii) an energy required for evaluating the training set, and / or (iv) an energy required for evaluating the validation set, and / or (v) a value of frames per second (FPS) of the candidate NNA, and / or (vi)Attorney Docket No.2010363-0434 one or more parameters of the candidate NNA. Evaluation based metrics may be associated with a combined model effectiveness. The combined model effectiveness is defined as: CM = AW × (Ta+Va) + FW × NF – EW × (TNE+VNE), where AW is the weight for accuracy, Tais the training accuracy, Vais the validation accuracy, FW is the weight for FPS, NF is a normalized FPS value, EW is the weight for energy, TNEis the normalized training energy, and VNEis the normalized validation energy.

[0099] FIG.4A shows a flowchart of an exemplary method for generating a neural network architecture tailored for a given task using a Large Language Model (LLM) and a Beam Search. The process begins in step 1 with the following inputs: (1) Base model (B) – the initial model from which modifications will be made; (2) Task specification (TS) to carry the information of the application (e.g., data size, number of output class of the task); (3) Training and validation datasets are used for training the model and evaluating its performance; (4) Beam width (BW) – a parameter that select the best models; (5) Initial modification (IM) – the initial set of changes applied to the base model to create variants; (6) Max variants (MV) – the maximum number of modifications allowed for the base model to improve its performance; and (7) Consecutive Poor Score (CPS) – a parameter that tracks the number of consecutive unsuccessful modifications.

[0100] In step 2, a Search Tree (ST) is initialized and a variant (V) is set to 1. In the next step, a variant of the base is generated by an LLM / AGI and the V is incremented by 1. In step 4, a generated model G is trained, an associated CM is calculated, and the CM and the G are both stored in the ST. In step 5, the method checks whether the number of variants has reached the initial modification limit (IM). If this limit has been reached, step 6 is executed, where the method checks whether the number of variants has reached the maximum variant (MV) limit. IfAttorney Docket No.2010363-0434 the number of variants is below the MV limit, then the method proceeds to step 7, where all models are sorted based on associated CM, and the BW models are selected and stored as selected models (SM). In step 8, an attempt is set to 1, taking the first model from the SM. In step 9, the model is modified using an LLM / AGI to a modified model (MM), the modified model is trained, and an associated CM is calculated. Next, the ST is updated with the MM and the associated CM. In step 11, the method checks if the associated CM of the MM is improved. If the associated CM is improved, the method proceeds to step 6 and the next iteration. If the associated CM is not improved, then the method increases the attempt by 1, tracking “failed” attempts to modify the model. In step 13, the method checks whether the attempt exceeds CPS. If the attempt exceeds CPS, the method proceeds to step 8 (taking a next model from the SM). Otherwise, the method proceeds to step 9 to further modify the model. The search process terminates once the number of variants reaches the maximum limit defined by the Max variants parameter.

[0101] FIG.4B shows a flowchart of another exemplary method for generating a neural network architecture tailored for a given task using a Large Language Model (LLM) and a Beam Search. At step 1, a user input is received, comprising of a base model (B), a task specification (TS), training and validation datasets, a beam width (BW), an initial modification count (IM), a maximum variant (MV) value, and a consecutive poor score (CPS). At step 2, a search tree (ST) is initialized and a number of variants is set to 1. At step 3, a variant G is generated using a first LLM. The first LLM receives a command based on the base model (B) and the user input and produces a variant G. The number of variants is increased by 1. At step 4, the generated variant G is trained and an associated CM and other evaluation metrics are determined and stored in ST. At step 5, a condition is examined whether the number of variants is greater than the initialAttorney Docket No.2010363-0434 modification count (IM). If it is, then proceed to step 6. Otherwise steps 3-5 are repeated to generate and train another variant as well as to determine and store an associated CM and other evaluation mertics. Step 6 represents a resulting search tree with every node representing a generated variant with associated CM and other evaluation metrics.

[0102] At step 7, a condition is examined whether the number of variants is greater than the maximum variant (MV) value. If it is, then the method is terminated and a variant with a highest CM is returned. Otherwise, the method proceeds to step 8. At step 8, an artificial intelligence (e.g., a neural network trained for the specific task) determines which variant out of the search tree based on the search tree is further modified. A number of attempts is set to 0. At step 9, a modified variant is created using a second LLM. The second LLM receives a command based on the determined variant, the user input, and associated evaluation metrics and produced a modified variant. In certain embodiments, the first and the second LLM as the same. In certain embodiments, the first LLM is different from the second LLM. The attempt is increased by 1. At step 10, the modified variant is trained and associated CM and other evaluation metrics are determined. The modified variant with the associated CM and other evaluation metrics are stored (e.g., appended as a separate node) in the ST. At step 11, a condition is examined whether the number of attempts exceeds CPS. If it does, then the method proceeds to step 7. Otherwise, steps 9-11 are repeated to generate and train another modified variant, determine associated CM and other evaluation metrics, and store them in the ST.

[0103] Algorithm 4 shows an exemplary algorithm for generating a neural architecture using a Beam Search. The Beam Search is a heuristic search algorithm that improves upon the best-first search. The Beam Search identifies the most promising node of a graph, which then serves as a reference for further expansion. The algorithm takes several inputs: (1) TaskAttorney Docket No.2010363-0434 Specification (TS) – metadata of the datasets; (2) MV – a maximum number of variants allowed for discovering the best neural network (NN); (3) Beam Width (BW) – a parameter that selects the best models; (4) Initial Command (Initial_Cmd) – a first prompt used to initiate the search for an NNA; (5) IM – an initial set of changes applied to the base model to create variants; and (6) CPS – a parameter that tracks the number of consecutive unsuccessful modifications.

[0104] The algorithm starts by initializing a best combined metric (BCM) and a best model (BestModel) as empty, sets a command (cmd) as the Initial Command, sets a variant to 1, and sets candidates as empty. In lines 7-15, an iterative process for generating models is described. In line 8, the algorithm sends the cmd to an LLM / AGI to generate models based on the requirements, subsequently creating a model from the response in line 9. In lines 10-14, the generated model is trained using a training set and then is evaluated with a validation set to obtain a validation accuracy and an inference time. Next, an associated CM is calculated using the validation accuracy and the inference time. The values of the validation accuracy, the inference time, the CM are all stored along with the model (its architecture). In lines 16-18, the generated models are sorted into a search tree based on associated CMs. The models of size BW are selected and the tree is updated. In lines 19-45, the sorted models are further modified, evaluated, and sorted in an iterative manner. In lines 20-23, the algorithm takes the associated values of model architecture, validation accuracy, CM and inference time from the sorted beam and initializes the updated as false. Best CM is set to CM and attempt is set to 0. In lines 24-43, while the attempt is below CPS and the variant is below MV, cmd is sent to the LLM / AGI to modify the model in line 26. In lines 27-30, a model is created, trained with the training set, and then evaluated with a validation set to obtain the validation accuracy (score) and the inference time. Next, CM is calculated and compared to the best CM to further modify the model based onAttorney Docket No.2010363-0434 the comparison. If the CM is improved, then the candidates are sorted, and the tree is updated. Otherwise, the attempt is increased by 1 in line 37 then algorithm proceeds to line 24. If the attempt reaches the CPS limit, then the algorithm proceeds to line 20 to take the second-best model from the tree for the next iteration. Algorithm 4. Neural Network Beam Search. 1: function BEAMSEARCH (TS, MV, BW, Initial_Cmd, IM, CPS) 2: BCM ← Ø8: Response ← LLM_AGI(cmd) 9: Model ← Create_Model(Response) 10: Model ← Model.train(TS.train_set) 11: val_acc, inf_time ← Model.eval(TS.val_set) 12: CM ← Effectiveness(val_acc, inf_time) 13: candidates←(Model, val_acc, CM, inf_time)) 14: variant+=1 15: end while 16: beam ← sorted(candidates, key=lambda x:x[2]) 17: beam ← beam[:BW] 18: Tree ← beam 19: while variant < MV do 20: for arch, score, CM, in f _time in beam do 21: updated = false 22: Best_CM ← CM 23: attempt ← 0 24: while attempt < CPS and variant < MV do 25: cmd ← (arch,score,inf_time) 26: Response ← LLM_AGI(cmd) 27: Model ← Create_Model(Response) 28: Model ← Model.train(TS.train_set) 29: val_acc, inf_time ← Model.eval(valset) 30: CM ← Effectiveness(val_acc, inf_time) 31: if CM > Best_CM then 32: Best_CM ← CMAttorney Docket No.2010363-0434 33: attempt ← 0 34: arch = model 35: updated = True 36: else 37: attempt ← +=1 38: end if 39: Tree.append(arch, Score, CM, inf_time) 40: / / sorted in ascending order 41: candidates ← SORTED((Tree, key=lambda x:x[2])) 42: beam ← candidates[:BW] 43: end while 44: end for 45: end while 46: return Tree 47: end function C. Software, Computer System, and Network Environment

[0105] Certain embodiments described herein make use of computer algorithms in the form of software instructions executed by a computer processor. In certain embodiments, the software instructions include a machine learning (ML) module, also referred to herein as artificial intelligence (AI) software. As used herein, a machine learning module refers to a computer implemented process (e.g., a software function) that implements one or more specific machine learning techniques, e.g., artificial neural networks (ANNs), e.g., convolutional neural networks (CNNs), random forest, decision trees, support vector machines, and the like, in order to determine, for a given input, one or more output values. In certain embodiments, the input comprises image data and / or alphanumeric data which can include 2D and / or 3D datasets, numbers, words, phrases, or lengthier strings, for example. In certain embodiments, the one or more output values comprise image data (e.g., 2D and / or 3D datasets) and / or values representing numeric values, words, phrases, or other alphanumeric strings.Attorney Docket No.2010363-0434

[0106] In certain embodiments, machine learning modules implementing machine learning techniques are trained, for example, using datasets that include categories of data described herein. Such training may be used to determine various parameters of machine learning algorithms implemented by a machine learning module, such as weights associated with layers in neural networks. In certain embodiments, once a machine learning module is trained, e.g., to accomplish a specific task such as identifying certain response strings, values of determined parameters are fixed and the (e.g., unchanging, static) machine learning module is used to process new data (e.g., different from the training data) and accomplish its trained task without further updates to its parameters (e.g., the machine learning module does not receive feedback and / or updates). In certain embodiments, available input data includes training data and validation data, e.g., where the validation data is separate and non-overlapping with the training data. For example, in certain embodiments, training data is used during the training process to optimize a model, whereas validation data is used to check the accuracy of the model while operating on previously unseen data. In certain embodiments, training data is divided into batches (e.g., portions) that is sequentially used (e.g., in random order) as sets of inputs to train a model. In certain embodiments, a model is trained multiple times (e.g., epochs) on the entire set of training data. In certain embodiments, machine learning modules may receive feedback, e.g., based on user review of accuracy, and such feedback may be used as additional training data, to dynamically update the machine learning module. In certain embodiments, two or more machine learning modules may be combined and implemented as a single module and / or a single software application. In certain embodiments, two or more machine learning modules may also be implemented separately, e.g., as separate software applications. A machine learning module may be software and / or hardware. For example, a machine learning module may be implementedAttorney Docket No.2010363-0434 entirely as software, or certain functions of a ANN module may be carried out via specialized hardware (e.g., via an application specific integrated circuit (ASIC) and / or field programmable gate arrays (FPGAs)).

[0107] In certain embodiments, machine learning modules implementing machine learning techniques may be composed of individual nodes (e.g., units, neurons). A node may receive a set of inputs that may include at least a portion of a given input data for the machine learning module and / or at least one output of another node. A node may have at least one parameter to apply and / or a set of instructions to perform (e.g., mathematical functions to execute) over the set of inputs. In certain embodiments, node instructions may include a step to provide various relative importance to the set of inputs using various parameters, such as weights. The weights may be applied by performing scalar multiplication (e.g., or other mathematical function) between a set of inputs values and the parameters, resulting in a set of weighted inputs. In certain embodiments, a node may have a transfer function to combine the set of weighted inputs into one output value. A transfer function may be implemented by a summation of all the weighted inputs and the addition of an offset (e.g., bias) value. In certain embodiments, a node may have an activation function to introduce non-linearity into the output value. Non-limiting examples of the activation function include Rectified Linear Activation (ReLu), logistic (e.g., sigmoid), hyperbolic tangent (tanh), and softmax. In certain embodiments, a node may have a capability of remembering previous states (e.g., recurrent nodes). Previous states may be applied to the input and output values using a set of learning parameters.

[0108] In certain embodiments, the machine learning module comprises a deep learning architecture composed of nodes organized into layers. For example, a layer is a set of nodes that receives data input (e.g., weighted or non-weighted input), transforms it (e.g., by carrying outAttorney Docket No.2010363-0434 instructions, e.g., applying a set of functions e.g., linear and / or non-linear functions), and passes transformed values as output (e.g., to the next layer). In certain embodiments, the set of nodes in a particular layer may share the same parameters and instructions without interacting with each other. A machine learning module may be composed of at least one layer (e.g., ordered). Examples of types of layers include convolutional layers (e.g., layers with a kernel, a matrix of parameters that is slid across an input to be multiplied with multiple input values to reduce them to a single output value); fully connected (FC) layers (e.g. all nodes are connected to all outputs of the previous layer); recurrent layers, long / short term memory (LSTM) layers, gated recurrent unit (GRU) layers (e.g., nodes with the various abilities to memorize and apply their previous inputs and / or outputs); batch normalization (BN) layers (e.g., layers that normalize a set of outputs from another layer, allowing for more independent learning of individual layers); activation layers (e.g., layers with nodes that only contain an activation function); and / or (un)pooling layers [e.g., layers that reduce (increase) dimensions of an input by summarizing (splitting) input values in defined patches).

[0109] In certain embodiments, the performance of a machine learning module may be characterized by its ability to produce an output data with specific accuracy. To achieve specific accuracy, a training process is performed to find optimal parameters, such as weights, for each node in each layer of the machine learning module. In certain embodiments, the training process of a machine learning module may involve using output data to calculate an objective function (e.g., cost function, loss function, error function) that needs to be optimized (e.g., minimized, maximized). For example, a machine learning objective function may be a combination of a loss function and regularization parameter. The loss function is related to how well the output is able to predict the input. The loss function may take various forms, like mean squared error, meanAttorney Docket No.2010363-0434 absolute error, binary cross-entropy, categorical cross-entropy, for example. The regularization term may be needed to prevent overfitting and improve generalization of the training process. Examples of regularization techniques include L1 Regularization or Lasso Regression, L2 Regularization or Ridge Regression, and Dropout (e.g., dropping layer outputs at random during training process).

[0110] In certain embodiments, objective function optimization of a machine learning module may involve finding at least one (e.g., all) of the present global optima (e.g., as opposed to local optima). In certain embodiments, the algorithm for objective function optimization follows principles of mathematical optimization for a multi-variable function and relies on achieving specific accuracy of the process. Examples of objective function optimization algorithms include gradient descent, nonlinear conjugate gradient, random search, Levenberg- Marquardt algorithm, limited-memory Broyden-Fietcher-Goldfarb-Shanno algorithm, pattern search, basin hopping method, Krylov method, Adam method, genetic algorithm, particle swarm optimization, surrogate optimization, and simulated annealing.

[0111] In certain embodiments, the machine learning modules comprise one of more generative AI modules. Rather than depending on use of predetermined weights and rules, generative AI leverages complex neural networks and algorithms to understand patterns and produce output that mimic human creativity. Examples of generative AI modules include image synthesis models (e.g., DALL-E3, DALL-E2, Imagen 3 in Gemini, Craiyon, and the like) and text generation models (e.g., ChatGPT, GPT-4, and the like).

[0112] In certain embodiments, the machine learning modules comprises one or more image-based segmentation neural networks. Illustrative examples of segmentation neuralAttorney Docket No.2010363-0434 networks include, for instance, Deep Image Matting (DIM), Semantic Segmentation methods (U- Net, DeepLab Series), Mask R-CNN, Chroma Keying CNNs, RefineNet, and MODNet.

[0113] In certain embodiments, AI used to generate alphanumeric text responsive to a user query and / or a set of input data may comprise (and / or utilize) one or more large language models (LLMs) [e.g., wherein the one or more LLMs comprise(s) one or more members selected from the group consisting of: BERT (Google) (or other transformer-based models), Falcon 40B, Galactica, GPT-3 (Generative Pre-trained Transformer, OpenAI), GPT-3.5 (OpenAI), GPT-4 (OpenAI), LaMDA (language model for dialogue applications, Google), Llama (large language model Meta AI) (Meta), Orca LLM (Microsoft), PaLM (Pathways Language Model), Phi-1 (Microsoft), StableLM (Stability AI), BLOOM (Hugging Face), RoBERTa (Meta), XLM- RoBERTa (Meta), NeMO LLM (Nvidia), XLNet (Google), Generate (Cohere), GLM-130B (Hugging Face), and Claude (Anthropic)] [e.g., wherein the one or more LLMs comprise(s) one or more members selected from the group consisting of an autoregressive LLM, autoencoding LLM, encoder-decoder LLM, bidirectional LLM, Fine-tuned LLMs, and multimodal LLMs].

[0114] As shown in FIG.5, an implementation of a network environment 500 for use in providing systems, methods, and architectures as described herein is shown and described. In brief overview, referring now to FIG.5, a block diagram of an exemplary cloud computing environment 500 is shown and described. The cloud computing environment 500 may include one or more resource providers 502a, 502b, 502c (collectively, 502). Each resource provider 502 may include computing resources. In some implementations, computing resources may include any hardware and / or software used to process data. For example, computing resources may include hardware and / or software capable of executing algorithms, computer programs, and / or computer applications. In some implementations, exemplary computing resources may includeAttorney Docket No.2010363-0434 application servers and / or databases with storage and retrieval capabilities. Each resource provider 502 may be connected to any other resource provider 502 in the cloud computing environment 500. In some implementations, the resource providers 502 may be connected over a computer network 508. Each resource provider 502 may be connected to one or more computing device 504a, 504b, 504c (collectively, 504), over the computer network 508.

[0115] The cloud computing environment 500 may include a resource manager 506. The resource manager 506 may be connected to the resource providers 502 and the computing devices 504 over the computer network 508. In some implementations, the resource manager 506 may facilitate the provision of computing resources by one or more resource providers 502 to one or more computing devices 504. The resource manager 506 may receive a request for a computing resource from a particular computing device 504. The resource manager 506 may identify one or more resource providers 502 capable of providing the computing resource requested by the computing device 504. The resource manager 506 may select a resource provider 502 to provide the computing resource. The resource manager 506 may facilitate a connection between the resource provider 502 and a particular computing device 504. In some implementations, the resource manager 506 may establish a connection between a particular resource provider 502 and a particular computing device 504. In some implementations, the resource manager 506 may redirect a particular computing device 504 to a particular resource provider 502 with the requested computing resource.

[0116] FIG.6 shows an example of a computing device 600 and a mobile computing device 650 that can be used to implement the techniques described in this disclosure. The computing device 600 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes,Attorney Docket No.2010363-0434 and other appropriate computers. The mobile computing device 650 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart- phones, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to be limiting.

[0117] The computing device 600 includes a processor 602, a memory 604, a storage device 606, a high-speed interface 608 connecting to the memory 604 and multiple high-speed expansion ports 610, and a low-speed interface 612 connecting to a low-speed expansion port 614 and the storage device 606. Each of the processor 602, the memory 604, the storage device 606, the high-speed interface 608, the high-speed expansion ports 610, and the low-speed interface 612, are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate. The processor 602 can process instructions for execution within the computing device 600, including instructions stored in the memory 604 or on the storage device 606 to display graphical information for a GUI on an external input / output device, such as a display 616 coupled to the high-speed interface 608. In other implementations, multiple processors and / or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system). Thus, as the term is used herein, where a plurality of functions are described as being performed by “a processor”, this encompasses embodiments wherein the plurality of functions are performed by any number of processors (one or more) of any number of computing devices (one or more). Furthermore, where a function is described as being performed by “a processor”, this encompasses embodiments wherein the function isAttorney Docket No.2010363-0434 performed by any number of processors (one or more) of any number of computing devices (one or more) (e.g., in a distributed computing system).

[0118] The memory 604 stores information within the computing device 600. In some implementations, the memory 604 is a volatile memory unit or units. In some implementations, the memory 604 is a non-volatile memory unit or units. The memory 604 may also be another form of computer-readable medium, such as a magnetic or optical disk.

[0119] The storage device 606 is capable of providing mass storage for the computing device 600. In some implementations, the storage device 606 may be or contain a computer- readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. Instructions can be stored in an information carrier. The instructions, when executed by one or more processing devices (for example, processor 602), perform one or more methods, such as those described above. The instructions can also be stored by one or more storage devices such as computer- or machine- readable mediums (for example, the memory 604, the storage device 606, or memory on the processor 602).

[0120] The high-speed interface 608 manages bandwidth-intensive operations for the computing device 600, while the low-speed interface 612 manages lower bandwidth-intensive operations. Such allocation of functions is an example only. In some implementations, the high- speed interface 608 is coupled to the memory 604, the display 616 (e.g., through a graphics processor or accelerator), and to the high-speed expansion ports 610, which may accept various expansion cards (not shown). In the implementation, the low-speed interface 612 is coupled to the storage device 606 and the low-speed expansion port 614. The low-speed expansion portAttorney Docket No.2010363-0434 614, which may include various communication ports (e.g., USB, Bluetooth®, Ethernet, wireless Ethernet) may be coupled to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or router, e.g., through a network adapter.

[0121] The computing device 600 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a standard server 620, or multiple times in a group of such servers. In addition, it may be implemented in a personal computer such as a laptop computer 622. It may also be implemented as part of a rack server system 624. Alternatively, components from the computing device 600 may be combined with other components in a mobile device (not shown), such as a mobile computing device 650. Each of such devices may contain one or more of the computing device 600 and the mobile computing device 650, and an entire system may be made up of multiple computing devices communicating with each other.

[0122] The mobile computing device 650 includes a processor 652, a memory 664, an input / output device such as a display 654, a communication interface 666, and a transceiver 668, among other components. The mobile computing device 650 may also be provided with a storage device, such as a micro-drive or other device, to provide additional storage. Each of the processor 652, the memory 664, the display 654, the communication interface 666, and the transceiver 668, are interconnected using various buses, and several of the components may be mounted on a common motherboard or in other manners as appropriate.

[0123] The processor 652 can execute instructions within the mobile computing device 650, including instructions stored in the memory 664. The processor 652 may be implemented as a chipset of chips that include separate and multiple analog and digital processors. TheAttorney Docket No.2010363-0434 processor 652 may provide, for example, for coordination of the other components of the mobile computing device 650, such as control of user interfaces, applications run by the mobile computing device 650, and wireless communication by the mobile computing device 650.

[0124] The processor 652 may communicate with a user through a control interface 658 and a display interface 656 coupled to the display 654. The display 654 may be, for example, a TFT (Thin-Film-Transistor Liquid Crystal Display) display or an OLED (Organic Light Emitting Diode) display, or other appropriate display technology. The display interface 656 may comprise appropriate circuitry for driving the display 654 to present graphical and other information to a user. The control interface 658 may receive commands from a user and convert them for submission to the processor 652. In addition, an external interface 662 may provide communication with the processor 652, so as to enable near area communication of the mobile computing device 650 with other devices. The external interface 662 may provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used.

[0125] The memory 664 stores information within the mobile computing device 650. The memory 664 can be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units. An expansion memory 674 may also be provided and connected to the mobile computing device 650 through an expansion interface 672, which may include, for example, a SIMM (Single In Line Memory Module) card interface. The expansion memory 674 may provide extra storage space for the mobile computing device 650, or may also store applications or other information for the mobile computing device 650. Specifically, the expansion memory 674 may include instructions to carry out or supplement the processes described above, and may include secure information also.Attorney Docket No.2010363-0434 Thus, for example, the expansion memory 674 may be provide as a security module for the mobile computing device 650, and may be programmed with instructions that permit secure use of the mobile computing device 650. In addition, secure applications may be provided via the SIMM cards, along with additional information, such as placing identifying information on the SIMM card in a non-hackable manner.

[0126] The memory may include, for example, flash memory and / or NVRAM memory (non-volatile random access memory), as discussed below. In some implementations, instructions are stored in an information carrier. The instructions, when executed by one or more processing devices (for example, processor 652), perform one or more methods, such as those described above. The instructions can also be stored by one or more storage devices, such as one or more computer- or machine-readable mediums (for example, the memory 664, the expansion memory 674, or memory on the processor 652). In some implementations, the instructions can be received in a propagated signal, for example, over the transceiver 668 or the external interface 662.

[0127] The mobile computing device 650 may communicate wirelessly through the communication interface 666, which may include digital signal processing circuitry where necessary. The communication interface 666 may provide for communications under various modes or protocols, such as GSM voice calls (Global System for Mobile communications), SMS (Short Message Service), EMS (Enhanced Messaging Service), or MMS messaging (Multimedia Messaging Service), CDMA (code division multiple access), TDMA (time division multiple access), PDC (Personal Digital Cellular), WCDMA (Wideband Code Division Multiple Access), CDMA2000, or GPRS (General Packet Radio Service), among others. Such communication may occur, for example, through the transceiver 668 using a radio-frequency. In addition, short-Attorney Docket No.2010363-0434 range communication may occur, such as using a Bluetooth®, Wi-Fi™, or other such transceiver (not shown). In addition, a GPS (Global Positioning System) receiver module 670 may provide additional navigation- and location-related wireless data to the mobile computing device 650, which may be used as appropriate by applications running on the mobile computing device 650.

[0128] The mobile computing device 650 may also communicate audibly using an audio codec 660, which may receive spoken information from a user and convert it to usable digital information. The audio codec 660 may likewise generate audible sound for a user, such as through a speaker, e.g., in a handset of the mobile computing device 650. Such sound may include sound from voice telephone calls, may include recorded sound (e.g., voice messages, music files, etc.) and may also include sound generated by applications operating on the mobile computing device 650.

[0129] The mobile computing device 650 may be implemented in a number of different forms, as shown in the figure. For example, it may be implemented as a cellular telephone 680. It may also be implemented as part of a smart-phone 682, personal digital assistant, or other similar mobile device.

[0130] Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.Attorney Docket No.2010363-0434

[0131] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the terms machine-readable medium and computer- readable medium refer to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term machine-readable signal refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0132] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0133] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination ofAttorney Docket No.2010363-0434 such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0134] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. EXAMPLES Example 1: Illustrative methodology for the automated generation of a neural network architecture tailored to perform a particular task

[0135] A high-level overview of an illustrative process for the automated generation of an NNA tailored for performing a particular user-defined task is depicted in FIG.7. This illustrative process is described below: 1. As an input to the framework, a user provides a description of a task to be performed. This information includes, for example: (1) nature of the application such as classification, regression, or the like; (2) an input data dimension; (3) an output prediction expected dimension; (4) a goal of the task; (5) a dataset to be used for training / validating the generated neural network architecture model; and (6) different application priorities that are to be used during the neural network generation process. Examples of the application priorities include: (a) a validationAttorney Docket No.2010363-0434 accuracy of a final neural network model must be greater than 90%; (b) maximizing validation accuracy is a first priority followed by improving energy efficiency during inferencing. 2. The Neural Network Generator Expert System (NNGES) generates a command to be fed to a Large Language Model (LLM) or an Artificial General Intelligence (AGI) model. This command contains information required to direct the generation of the required neural network model requested by the user of the framework. The command is generated based on the input provided, a set of rules (Neural Network Generator Rule Book), and a history of this generation process (earlier models predicted and their corresponding evaluation reports). 3. Next, the command is fed to the LLM / AGI. The LLM / AGI provides a candidate Generated Neural Network Architecture (NNA). 4. Next, the Generated Neural Network Architecture (NNA) is evaluated by partially training it and validating it using a subset of the user-provided dataset. Different metrics are tracked during this evaluation. 5. The most optimal NNA among the candidate NNAs generated across multiple iterations is maintained. 6. After that, if the process limit of the generation routine is deemed to have been reached, then the most optimal NNA (trained) is provided as the output of the generation process. Otherwise, steps 2, 3, 4, 5, and 6 are repeated to find a more optimal NNA. Illustrative Results

[0136] The framework for designing neural network architectures is utilized for different datasets and with different settings (user-defined metrics) summarized in Table 3. Table 4 shows the results for CIFAR-10 dataset. The CIFAR-10 dataset consists of 60,00032x32 color images in 10 classes, with 6000 images per class – there are 50,000 training images and 10,000 testAttorney Docket No.2010363-0434 images. Further details about the CIFAR-10 dataset are available at https: / / www.cs.toronto.edu / ~kriz / cifar.html, the text of which is incorporated by reference herein in its entirety. The generated neural network architectures reflect the user-defined priorities. For example, when accuracy is prioritized, a validation accuracy of 90.9% is achieved, which is a very high value for this dataset. Similar results for CIFAR-100 are presented in Table 5. Table 3. User-defined parameters used for different experimental settings. Setting PA1PA2PE1PE2PF PP 1 0.4 0.6 0 0 0 0Attorney Docket No.2010363-0434 Table 4. Effectiveness of the described framework for generating optimal neural network models for CIFAR-10 dataset using ChatGPT4.0 as the LLM / AGI. Set- ting 1 2 3 4 5. . . . . . . Table 5. Effectiveness of the described framework for generating optimal neural network models for CIFAR-100 dataset using ChatGPT4.0 as the LLM / AGI. Set- ting 1 2 3 45 63.90% 3489316 4.68 0.73515 0.7013343 10026.15 7.44E-06 7.10E-06Attorney Docket No.2010363-0434

[0137] FIG.8A shows a model generated by LLM (GPT-4 Turbo) for CIFAR-10 dataset where a full priority (1) was given on the validation accuracy (setting 3 in Table 3). This model showed 90.90% validation accuracy and 89.41% test accuracy, taking 4.25 hours including searching the architecture and final training the model. To search the architecture and training final model, the method consumed 0.704 kWh power utilization effectiveness (PUE) which is equivalent to 0.672 lb. CO2 emission. The total number of parameters of this model is 699466.

[0138] FIG.8B shows the model generated by LLM (GPT-4 Turbo) for CIFAR-10 dataset, where equal priority was given to training and validation energy consumption, i.e., 0.5 for training and 0.5 for validation (setting 4 in Table 3). This model showed 70.21% validation accuracy and 69.26% test accuracy, taking 4.84 hours including searching the architecture and final training the model. The total number of parameters for this model is 136874, which is lower as compared to the model in FIG.4A because of giving priority on energy budget. Example 2: Illustrative results for the automated generation of a neural network architecture tailored to perform a particular task

[0139] This section provides an illustrative embodiment of a method for automated generation of a neural network architecture tailored to perform a particular task, called Large Language Model guided Neural Architecture Discovery (LeMo-NADe), implemented using exemplary algorithms Algorithm 1, Algorithm 2, and Algorithm 3.

[0140] Five publicly available datasets CIFAR-10, CIFAR-100, ImageNet16-120, EuroSAT, and Malaria Parasite were used to validate LeMo-NADe. All experiments were run on a single NVIDIA A100 GPU to make a fair determination of metrics such as power consumptionAttorney Docket No.2010363-0434 and runtime. A python library named PyJoules was used to measure the energy consumption of both CPU and GPU during network search / training.

[0141] Both the CIFAR-10 and CIFAR-100 datasets comprise 60k images, each with dimensions of 32×32 pixels where 50k and 10k samples are designated for training and testing purposes, respectively. The CIFAR-10 dataset has 10 output classes whereas the CIFAR-100 datasets have 100 output classes. ImageNet16-120 has 151k training and 6k testing samples with a resolution of 16×16 having 120 classes. The Malaria parasite dataset has two classes, that is, (i) parasitized cells and (ii) uninfected cells with 27558 data samples in total. The data sets were resized to 32x32 resolution. Next, the datasets were split into an 8:2 ratio for the training and validation sets. And finally, the EuroSAT dataset contains 27000 data samples of 10 classes, i.e., (i) AnnualCrop (ii) Forest (iii) HerbaceousVegetation (iv) Highway (v) Industrial (vi) Pasture (vii) PermanentCrop (viii) Residential (ix) River, and (x) SeaLake. The dataset was resized to 32x32 resolution and was split into 8:2 ratios for training and validation sets.

[0142] During the neural network generation process, LeMo-NADe was executed for 30 iterations. The resulting network was subsequently trained for 50 epochs with a batch size of 128, utilizing Stochastic Gradient Descent (SGD) optimizer along with an initial learning rate of 0.025 and a weight decay parameter set to 3×10−4. To enhance the convergence rate, a cosine annealing learning rate schedule was employed, which modulates the learning rate according to a cosine function. Furthermore, to mitigate overfitting, data augmentation techniques such as random rotation by 10 degrees, random horizontal flipping, and random cropping of size 16x16 were integrated into the training regimen to facilitate diverse learning. A final model was trained on the entire dataset over 600 epochs. A batch size of 256 was employed, utilizing the SGD optimizer, with the initial learning rate set at 0.025 accompanied by a weight decay of 3×10−4.Attorney Docket No.2010363-0434 The learning rate is systematically adjusted according to the annealing learning rate schedule. For LLM, GPT-4o was configured with a temperature parameter of 0.5.

[0143] Table 6 provides a comparative analysis of several methods for neural network generation in conjunction with LeMo-NADe, utilizing the CIFAR-10, CIFAR-100, and ImageNet16-120 datasets. The analysis is concentrated exclusively on accuracy, as it represents the primary objective for all the methods under evaluation. As indicated in the table, the CGP- CNN method required 744 GPU hours for the neural architecture search (NAS) process, achieving accuracies of 94.05% and 73.3% on the CIFAR-10 and CIFAR-100 datasets, respectively. Conversely, the GeNet approach utilized 408 GPU hours to identify the optimal model, resulting in accuracies of 92.9% for CIFAR-10 and 70.95% for CIFAR-100. These methods are notably resource-intensive, necessitating substantial computational time to identify the optimal model. However, several methods have emerged, requiring significantly less time while delivering relatively competitive performance: DrNAS required only 1.2 GPU hours to achieve accuracies of 94.36%, 73.51%, and 46.34% on the CIFAR-10, CIFAR-100, and ImageNet16-120 datasets, respectively, whereas RMI expended merely 0.34 GPU hours, yielding 94.28±0.10%, 73.36±0.19%, and 46.34±0.00% accuracies on the CIFAR-10, CIFAR-100, and ImageNet16-120 datasets, respectively. The LeMo-NADe method required 5.8 and 7.12 GPU hours to find the optimal architecture for the CIFAR-10 and CIFAR-100 datasets, achieving accuracies of 95.51% and 79.43%, demonstrating state-of-the-art results among all other NAS methods. For the ImageNet16-120 dataset, the LeMo-NADe method attained an accuracy of 42.95% within 12.83 GPU hours, approaching closely to the state-of-the-art performance. Table 6. Performance Metrics for Various Methods Across Datasets.Attorney Docket No.2010363-0434 Methods Cost Accuracy (%)(GPU Hours) CIFAR-10 CIFAR-100 ImageNet16-120DeepMaker 75 93.1 75.13 N / A 94.05 73.3 N / AEIGEN 94.6 N / A N / AN / A 78.1 N / A N / AGeNet 408 92.9 70.95 N / A NSGANet 648 95.33 74.83 N / A NASHBOT 40.8 91.31 N / A N / A NASH-Net 24 94.8 N / A N / A GDAS 8.7 93.61 70.3 41.71 DARTS- 3.2 93.8 71.53 45.12 GENIUS N / A 93.79±0.09 70.91±0.81 44.96±1.02 DrNAS 1.2 94.36 73.51 46.34 SE-NAS 2.93 93.47±0.14 N / A 45.66±1.05 FairNAS 2.73 93.23±0.18 N / A 42.19±0.31 Shapley-NAS 7.2 94.37±0.00 N / A 46.85±0.12 Distribution Constrained3.9 94.29±0.07 N / A 46.41±0.14FreeRea N / A 94.36 N / A 46.34 RMI 0.34 94.28±0.10 73.36±0.19 46.34±0.00 LeMo-NADe 5.8 95.51 - - (Proposed) 7.12 - 79.43 - 12.83 - - 42.95

[0144] FIGs.9A-9B show graphs of the training and validation accuracy and loss of five different datasets during final training with 600 epochs.

[0145] The following goodness metric is proposed: GoodNess =WA×Accuracy+(1−ENorm)×WEwhere WAand WEare weights of accuracy and energy, respectively, and ENormis a normalized energy obtained as: ^^^^ − ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^= ,Attorney Docket No.2010363-0434 where, E, Eminand Emaxare energy, maximum energy and minimum energy obtained from the Table 7, respectively. The architecture search cost (energy) is considered along with accuracy for an objective comparison between all NAS methods. The Energy is calculated as a measure of kilowatt-hour power utilization effectiveness (kWh-PUE): pt= 1.58t × (pc+ pr+gpg) / 1000, where pc, pr, and pgrepresent the power usages (in watt) of CPU, RAM and GPU, respectively, t is the total run time in hours and g is the number of GPUs. The run time for each NAS method and the maximum GPU power draw (in watt) of each GPU were taken from Table 6.

[0146] Table 7 shows that for the case of considering weight of accuracy 0.8 and weight of energy 0.2, CGP-CNN shows 0.752 goodness metric although it showed 94.05% accuracy (Table 6). This is because of the associated excessive search cost. Similar results are also noticeable for GeNet and NSGANet also for the same issue. The Lemo-NADe outperforms all other NAS methods for CIFAR-10 and CIFAR-100, while achieving nearly equal goodness metric with other NAS for ImageNet16-120.Attorney Docket No.2010363-0434 Table 7. Comparing GoodNess metric of different methods and datasets with various configuration. Me Deep CGP EIGe NSG NAS NAS G DA Dr SE- Fair Sha N D R LeMo (Pro. . . .Attorney Docket No.2010363-0434

[0147] For another configuration (weight of accuracy and energy is equal i.e., both are 0.5), the Lemo-NADe approach outperforms all other methods for CIFAR-10 and CIFAR-100, while ImageNet16-120’s metric is comparable. Finally, for the third configuration, weight of accuracy 0.2 and weight of energy 0.8, the LeMo-NADe outperforms all NAS methods for CIFAR-10 and for CIFAR-100.

[0148] LeMo-NADe was executed in diverse configurations throughout the search process. The experimental results are documented in Table 8. The table illustrates that when LeMo-NADe prioritized accuracy exclusively (assigning a weight of 1 to accuracy), it exhibited enhanced performance with respect to accuracy. Specifically, for CIFAR-10, a complete focus on accuracy yielded an accuracy of 95.51%, contrasting with an accuracy of 92.72% when equal emphasis was placed on accuracy and energy. Similar trends are observable across the other four datasets. Table 8. Performance Metrics of the proposed method for Various Datasets Under Different Settings. Datasets Settings CostEnergy Accuracy (Hours)(kWh-PUE) (%)Attorney Docket No.2010363-0434 WA= 0.7, WE= 0.1, WF= 0.2 4.3 0.620 97.48o t e be av or o G - o durng ts searc process s urt er provided, with particular emphasis on its interaction with the expert system of LeMo-NADe. FIG.10A shows how GPT-4o handles feedback from the expert system using the before and after neural network architecture representations. Case 1: The initially generated model was a simple CNN model with two convolutional layers having ReLU activations and Batch Normalization (BN), Maxpooling, and a single Dense layer. After getting a feedback (ACL,ASC,ADL), GPT-4o generated a model with four blocks of convolutional layers with ReLU activation and Batch Normalization (BN), and two Dense layers, also adding two skip connections based on the feedback. Case 2 shows another behavior of GPT-4o where it added more skip connections by adding some convolutional layers. Case 3: in some situations, skip connections make the network architecture more energy intensive to train / run. GPT-4o reduces the skip connections based on the feedback from the expert system. Case 4: GPT-4o not always showed outstanding performance by generating models. The responses from GPT-4o were carefully checked to observe that in approximately 10% of the cases GPT-4o fails to follow the provided instructions. For example, in one case, the expert system suggested GPT-4o to add more skip connections, but it failed to do so. In another case, GPT-4o generated an architecture, which was not possible because of a mismatch in a shape of the layers. Example 3: Illustrative results for the automated generation of a neural network architecture tailored to perform a particular taskAttorney Docket No.2010363-0434

[0150] This section provides an illustrative embodiment of a method for automated generation of a neural network architecture tailored to perform a particular task, called Large Language Model guided Neural Architecture Discovery (LeMo-NADe), implemented using exemplary algorithms Algorithm 1, Algorithm 2, and Algorithm 3.

[0151] Three publicly available datasets CIFAR-10, CIFAR-100, and ImageNet16-120 are used. All experiments are run on a single NVIDIA A100 GPU (to make a fair determination of metrics such as power consumption and runtime). Both the CIFAR-10 and CIFAR-100 datasets comprises 60k images, each with dimensions of 32 × 32 pixels where 50k and 10k samples are designated for training and testing purposes respectively. The CIFAR-10 dataset have 10 output classes where CIFAR-100 datasets have 100 output classes. The training data was split in a 9:1 ratio for generating the new training and validation set respectively. ImageNet16-120 has 151k training and 6k testing samples with a resolution of 16 × 16 having 120 classes. The training dataset was again split in a 9:1 ratio towards creating a new training and validation set. During neural discovery, the LeMo-NADe was run for 30 iterations and then the generated network was trained for 20 epochs using batch size of 512 (Adam optimizer and 0.001 learning rate). The learning rate was reduced by 10% of its initial value if validation loss have not reduced for 5 successive epochs. To reduce overfitting, the training dataset was augmented with random rotation of 20 degrees, random horizontal flip, 10% random width shift, 10% random height shift, 10% random shear, and 10% random zoom.

[0152] After LeMo-NADe determined the most optimal neural network, the network was retrained for 500 epochs (max) using a batch size of 1024. The Adam optimizer was utilized, starting with an initial learning rate of 0.01. This learning rate was decreased if there is noAttorney Docket No.2010363-0434 reduction in validation loss over 20 consecutive epochs. The GPT-4 Turbo with temperature setting of 0 was used.

[0153] Table 9 shows the experimental results for CIFAR-10 dataset (using settings described in Table 3). For GPT-4 Turbo, better test accuracy of 89.41% was obtained when the validation accuracy priority is set to 1. For this case, the total run time (searching time + model training) is 4.25 hours, the power utilization effectiveness (PUE) is 0.7041 kWh, and CO2 emission is 0.6717 lb. Since both training and inferencing energy are prioritized in setting 4, LeMo-NADe generated a much simpler neural network while sacrificing some validation accuracy (69.26%). The PUE and CO2 emission were calculated using the following two equations: pt= 1.58t × (pc+ pr+gpg) / 1000, CO2e = 0.954 × pt, where pc, pr, and pgrepresent the power usages (in watt) of CPU, RAM and GPU, respectively, t is the total run time in hours and g is the number of GPUs. For Gemini, the highest test accuracy of 81.76% was obtained for setting 1, where LeMo-NADe prioritized training accuracy and validation accuracy. This generation (and training) took 2.14 hours and consumed 0.1605 kWh- PUE of energy emitting 0.1531 lb of CO2. The efficient model for inferencing (in terms of energy consumed) was obtained for setting 4 where LeMo-NADe gave equal priority to training and inferencing time energy consumption. For this setting, LeMo-NADe took 2.87 hours (generating and training time), consumed 0.1811 kWh-PUE, and emitted 0.1728 lb CO2. Overall, the GPT-4 Turbo perform better compared to Gemini.Attorney Docket No.2010363-0434 Table 9. Effectiveness of LeMo-NaDe for generating optimal neural network models for CIFAR-10 dataset. GENERA- GENERA- GENERA- INFEREN- TING INFEREN- INFEREN- TING TING CING NUMBER + CING CING SET- TEST + + ENERGY LLM OF TRAINING FRAME CO2 TING ACCURACY TRAINING TRAINING PER PARAMS CO2 PER EMISSION RUNTIME ENERGY IMAGE EMISSION SECOND (LBS) (HOURS) (KWH-PUE) (KWH-PUE) (LBS) 1 82.55 % 2442698 3.93 0.5157 0.4920 6897.22 1.11E-05 1.06E-05 GPT-4 2 84.27% 357706 4.60 0.5682 0.5421 115220.68 5.34E-06 5.09E-06 TURBO 3 89.41 % 699466 4.25 0.7041 0.6717 10755.78 7.06E-06 6.74E-06 4 69.26 % 136874 4.84 0.4751 0.4533 14636.72 5.39E-06 5.14E-06 5 85.44 % 2400330 4.27 0.5743 0.5480 13051.73 5.89E-06 5.62E-06 1 81.76 % 440778 2.14 0.1605 0.1531 13435.93 3.18E-06 3.04E-06 GEMI- 2 81.20 % 128842 2.79 0.2333 0.2226 21553.23 2.44E-06 2.33E-06 NI 3 80.79 % 3016522 2.403 0.2064 0.1969 20460.87 1.08E-06 1.03E-06 4 71.73 % 79530 2.87 0.1811 0.1728 23029.86 7.42E-07 7.08E-07 5 78.59 % 1276234 1.94 0.1264 0.1206 19565.53 6.42E-06 6.12E-06Attorney Docket No.2010363-0434

[0154] Table 10 presents the outcomes of experiments performed with the CIFAR-100 dataset, following the same parameters outlined in Table 3. GPT-4 Turbo showed superior results regarding testing accuracy (67.90%), lower number of model parameters (79530), lower generating + training energy (0.5021 kWh-PUE), and higher FPS (15079.86) for setting 4. The generated model has 64.55% test accuracy when 0.3, 0.3, and 0.4 were provided as the priorities for FPS, training accuracy and validation accuracy, respectively. For this model, image inferencing took 7.44 × 10−6 kWh-PUE energy. Gemini performs better for setting 2 in terms of test accuracy, parameter, runtime and energy consumption. This generated model had test accuracy of 52.96%, took 2.48 hours for generating / training, and consumed 0.0889 kWh-PUE which is least among all other settings. Setting 3 leads to a test accuracy of 50.20% where full priority is provided to the validation accuracy and it took 2.65 hours for generation / training.Attorney Docket No.2010363-0434 Table 10. Effectiveness of LeMo-NaDe for generating optimal neural network models for CIFAR-100 dataset. GENERA- GENERA- GENERA- TING INFEREN- INFEREN- INFEREN- TING TING TEST NUMBER + CING CING CING + + LLM SETTING ACCU- OF TRAINING FRAME ENERGY CO2 TRAINING TRAINING RACY PARAMS CO2 PER PER IMAGE EMISSION RUNTIME ENERGY EMISSION SECOND (KWH-PUE) (LBS) (HOURS) (KWH-PUE) (LBS) 1 56.94% 948708 4.46 0.6874 0.6558 11753.98 7.95E-06 7.59E-06 GPT-4 2 63.43% 1725348 4.73 0.7142 0.6813 10278.36 6.08E-06 5.80E-06 TURB 3 58.49% 948708 4.13 0.6056 0.5778 13294.64 5.96E-06 5.68E-06 O 4 67.90% 79530 4.81 0.5021 0.4790 15079.86 5.63E-06 5.37E-06 5 64.55% 3489316 4.68 0.7351 0.7013 10026.15 7.44E-06 7.10E-06 1 47.78 % 721060 2.21 0.1609 0.1535 21455.82 2.91E-06 2.77E-06 52.96 % 801444 2.48 0.0932 0.0889 20740.8 2.48E-06 2.36E-06 NI 3 50.20 % 4385540 2.65 0.2877 0.2744 18524.72 2.67E-06 2.55E-06 4 10.24 % 12836 2.83 0.1015 0.097 20726.49 1.19E-06 2.09E-06 5 51.33 % 2665124 2.92 0.2115 0.2018 18470.16 1.04E-06 9.90E-07Attorney Docket No.2010363-0434

[0155] Table 11 presents the outcomes of studies experiments carried out using the ImageNet16-120 dataset. LeMo-NADe with GPT-4 Turbo shows 27.70% test accuracy for setting 3 which is the best accuracy among all settings. It takes 7.21 hours to generate and train the final model with an energy consumption of 0.7476 kWh-PUE. For setting 1 (0.4 priority on training accuracy and 0.6 priority on validation accuracy), a test accuracy of 27.43% and a generation / training energy consumption of 0.6476 kWh-PUE was obtained. Gemini also showed promising performance in terms of accuracy and model parameters, requiring low training / inferencing energy. As seen in Table 11, LeMo-NADe using Gemini shows 31.02% accuracy for setting 2 with comparatively lower number of model parameter (368440) and having a training energy of 0.4680 kWh-PUE. For this dataset, ImageNet16-120, Gemini outperforms GPT-4 Turbo.Attorney Docket No.2010363-0434 Table 11. Effectiveness of LeMo-NaDe for generating optimal neural network models for ImageNet16-120 dataset. GENERA- GENERA- GENERA- TING INFEREN- INFEREN- INFEREN- TING TING TEST NUMBER + CING CING CING SET- + + LLM ACCU- OF TRAINING FRAME ENERGY CO2 TING TRAINING TRAINING RACY PARAMS CO2 PER PER IMAGE EMISSION RUNTIME ENERGY EMISSION SECOND (KWH-PUE) (LBS) (HOURS) (KWH-PUE) (LBS) 1 27.43% 1476536 6.26 0.6476 0.6178 13027.34 1.09E-05 1.04E-05Attorney Docket No.2010363-0434

[0156] Table 12 compares different leading-edge NAS techniques (that used predefined search spaces to identify the optimal network) and LeMo-NADe on CIFAR-10, CIFAR-100 and ImageNet16-120 datasets. For CIFAR-10 dataset, LeMo-NADe shows promising results and in some cases, it outperforms DARTS, DSNAS, PC-DARTS, iDARTS, and GDAS. For CIFAR- 100 dataset, LeMo-NADe with GPT-4 Turbo outperforms DARTS, DSNAS, and PC-DARTS for both validation and test set results. For ImageNet16-120, LeMo-NADe with Gemini perform better than LeMo-NADe with GPT-4 Turbo, showing 30.34% and 31.02% accuracy for validation and test set, respectively, which is approximately twice better than the DARTS method.Attorney Docket No.2010363-0434 Table 12. Comparison between LeMo-NaDe and other NAS frameworks. METHOD CIFAR-10 CIFAR-100 IMAGENET16-120TEST VALIDATION TEST VALIDATION TEST39.77 54.30 38.57 15.61 18.87 16.32 89.66 30.87 30.87 31.01 40.61 41.07 PC- 89.96 93.41 67.12 67.48 40.83 41.31 89.86 93.58 70.57 70.83 40.38 40.89 89.89 93.61 71.34 70.70 41.59 41.71 β - 91.55 94.36 73.49 73.51 46.37 46.34 Λ - 91.55 94.36 73.49 73.51 46.37 46.34 91.07 93.79 70.96 70.91 45.29 44.96 90.90 89.41 68.38 67.90 27.05 27.7082.94 81.76 52.12 52.96 30.34 31.02Attorney Docket No.2010363-0434

[0157] FIGS.10B-10C show the behavior of GPT-4 Turbo (cases 1-4) and Gemini (cases 5-8) based on the feedback from the Expert System. Case 1: The initially generated model was a simple CNN model with two convolutional layers having ReLU activations and Batch Normalization (BN), Maxpooling, and a single Dense layer. After getting the feedback (ACL,ASC,ADL), GPT-4 Turbo generated a model with four blocks of convolutional layers with ReLU activation and Batch Normalization (BN), and two Dense layers. GPT-4 Turbo also added two skip connections based on the feedback. Case 2 shows another behavior of GPT-4 Turbo where it added more skip connection by adding some convolutional layers. Case 3: In some situations, skip connections make the network architecture more energy intensive to train / run. In this case, GPT-4 Turbo reduced the skip connections based on the feedback from ES. Case 4: GPT-4 Turbo does not always show outstanding performance. In approximately 15% of the cases, GPT-4 Turbo failed to follow the provided instructions. For example, in this case, GPT-4 Turbo suggested a model with improper shape. Case 5: Gemini failed to build the network because of the negative dimensions produced. This happened because of using convolution layers with the maxpooling layers (having a larger filter size). Case 6: Gemini reduced a skip connection by taking a decision itself (it also describes the intuition behind this: reduce skip connection because accuracy doesn’t improve by using this) although LeMo-NADe suggested adding more skip connections, convolutional layers, and dense layers. Case 7: Gemini added more convolutional layers with more skip connection as per the instructions given by LeMo- NADe. Case 8: Although Gemini showed promising results, in about 25% of the cases, it failed to generate a valid model because of negative dimensions and shape mismatches (when trying to concatenate with another layer). Gemini attempted to correct errors during next iteration.Attorney Docket No.2010363-0434

[0158] Tables 13-15 show detailed experimental results of LeMo-NADe when using GPT-4 Turbo on CIFAR-10, CIFAR-100 and ImageNet16-120 datasets, respectively. The temperature values were varied within the range of 0.2 to 1 and evaluated the generated neural network architecture. Table 13 shows that LeMo-NADe performs better for Temperature 0.6 in terms of accuracy, model parameter, computing energy and inference FPS. On the other hand, Table 14 shows that LeMo-NADe with GPT-4 Turbo shows better performance for Temperature value 0.8. LeMo-NADe also shows better results for Temperature 0.8 for ImageNet16-120 dataset as reported in Table 15.

[0159] FIG.11 illustrates a model generated by LeMo-NADe with GPT-4 Turbo for CIFAR-10 dataset. This model is obtained for the following configuration: Temperature 0.6; Priorities: Training accuracy: 0, Training energy: 0, Validation accuracy: 1, Validation energy: 0, FPS: 0; Threshold: Training accuracy: 0.95, Validation accuracy: 0.92, FPS: 20000, Training energy: 10−3kWh, Validation energy: 10−5kWh.

[0160] FIG.12 illustrates a model generated by LeMo-NADe with GPT-4 Turbo for CIFAR-100 dataset. This model is obtained for the following configuration: Temperature 0.4; Priorities: Training accuracy: 0.4, Training energy: 0, Validation accuracy: 0.6, Validation energy: 0, FPS: 0; Threshold: Training accuracy: 0.90, Validation accuracy: 0.85, FPS: 20000, Training energy: 10−2kWh, Validation energy: 10−4kWh.

[0161] FIG.13 illustrates a model generated by LeMo-NADe with GPT-4 Turbo for ImageNet16-120 dataset. This model is obtained for the following configuration: Temperature 0.4; Priorities: Training accuracy: 0.4, Training energy: 0, Validation accuracy: 0.6, Validation energy: 0, FPS: 0; Threshold: Training accuracy: 0.70, Validation accuracy: 0.65, FPS: 22000, Training energy: 10−3kWh, Validation energy: 10−5kWh.Attorney Docket No.2010363-0434 Table 13. Effectiveness of LeMo-NaDe for generating optimal neural network models for CIFAR-10 dataset using GPT-4 Turbo with different temperature values. TEMP SE1 84.42% 3491146 4.10 0.5545 0.5290 12009.58 7.50E-07 7.15E-06 2 85.06% 621258 4.59 0.5794 0.5528 7461.53 5.08E-06 4.85E-06 T = 0.2 3 85.65% 3494730 4.41 0.8478 0.8088 10904.14 7.15E-06 6.82E-06 4 66.25% 795330 4.75 0.4531 0.4323 15335.26 5.28E-06 5.04E-06 5 87.32% 4605642 4.05 0.5374 0.5127 13167.46 6.09E-06 5.81E-06 1 83.37% 3491146 3.99 0.6099 0.5819 11487.42 1.09E-06 1.04E-06 2 84.03% 621258 4.08 0.5011 0.4781 13521.36 5.66E-06 5.40E-06 T = 0.4 3 87.59% 3597002 3.89 0.6717 0.6408 108132.38 7.24E-06 6.91E-06 4 65.65% 79530 4.70 0.5086 0.4852 15032.04 5.93E-06 5.66E-06 5 89.02% 3418570 4.27 0.5725 0.5461 10377.16 1.09E-05 1.04E-05 1 89.70% 3428938 3.92 0.5068 0.4834 11013.01 1.03E-05 9.81E-06 2 84.11% 620746 4.15 0.4889 0.4664 12560.41 5.02E-06 4.79E-06 T = 0.6 3 90.90% 5884042 4.05 0.7806 0.7447 9218.27 7.41E-06 7.07E-06 4 65.14% 79530 4.65 0.4436 0.4232 15426.29 2.63E-06 2.51E-06 5 88.67% 349530 4.20 0.5266 0.5024 11622.50 9.99E-06 9.54E-06 1 89.13% 4997898 4.84 2.09 2.0 4743.40 1.31E-05 1.25E-05 2 83.28% 361930 3.57 0.4518 0.4301 12494.94 5.54E-06 5.28E-06 T = 0.8 3 89.53% 4778762 3.99 0.6022 0.5745 10870.27 6.61E-06 6.30E-06 4 63.48% 231562 4.66 0.4759 0.4541 15866.69 2.78E-06 2.66E-06 5 90.46% 9675082 3.61 0.4646 0.4432 11438.98 1.01E-05 9.60E-06 1 82.22% 2401866 3.67 0.4656 0.4442 11855.67 9.06E-06 8.63E-06 T = 1 2 86.34% 9188298 4.24 0.5778 0.5512 12245.07 6.56E-06 6.26E-06 3 84.55% 3510858 3.78 0.5030 0.4798 12511.31 6.82E-06 6.50E-06Attorney Docket No.2010363-0434 4 67.17% 79530 4.69 0.4811 0.4589 14869.92 5.89E-06 5.62E-06161098 3.77 0.4118 0.3929 14786.20 5.85E-06 5.58E-06Attorney Docket No.2010363-0434 Table 14. Effectiveness of LeMo-NaDe for generating optimal neural network models for CIFAR-100 dataset using GPT-4 Turbo with different temperature values. TE1 56.43% 3489316 4.50 0.8881 0.8472 9389.76 1.07E-05 1.02E-05 2 58.03% 948708 4.18 0.5707 0.5444 12694.01 5.11E-06 4.88E-06 T = 0.2 3 58.85% 948708 4.19 0.8080 0.7709 12374.44 5.72E-06 5.46E-06 4 45.19% 635172 4.77 0.4730 0.4512 15020.50 5.35E-06 5.11E-06 5 57.67% 948708 4.57 0.7865 0.7503 13518.15 5.32E-06 5.07E-06 1 57.74% 948708 4.21 0.7334 0.6997 11285.20 5.38E-06 5.13E-06 2 62.42% 463380 4.25 0.6635 0.6330 11060.66 6.19E-06 5.90E-06 T = 0.4 3 53.45% 4630308 4.26 0.7017 0.6694 10563.94 5.64E-06 5.38E-06 4 45.28% 635172 4.71 0.4597 0.4385 14895.69 7.96E-06 7.59E-06 5 61.90% 34166180 4.62 0.8843 0.8436 11405.19 1.08E-05 1.03E-05 1 56.22% 1543844 3.95 0.6204 0.5919 9703.77 7.62E-06 7.27E-06 2 59.34% 948708 4.05 0.6515 0.6215 13147.95 5.96E-06 5.69E-06 T = 0.6 3 44.92% 6108708 4.38 0.7732 0.7377 11135.41 6.73E-06 6.42E-06 4 46.52% 635172 4.74 0.4518 0.4310 14760.20 5.12E-06 4.89E-06 5 60.44% 3441700 3.99 0.6337 0.6046 12014.46 7.04E-06 6.71E-0 1 54.60% 4624932 3.99 0.5776 0.5510 11605.52 5.69E-06 5.43E-06 2 63.69% 2674852 4.42 0.6246 0.5958 12606.06 5.73E-06 5.46E-06 T = 0.8 3 50.70% 3557284 3.88 0.6374 0.6080 12400.02 1.04E-05 9.92E-06 4 40.80% 325956 4.72 0.4644 0.4431 14583.39 2.61E-06 2.49E-06 5 67.52% 6009124 4.36 0.6583 0.6280 10632.62 1.10E-05 1.05E-05 T = 1 1 51.60% 6166756 3.89 0.5683 0.5421 11330.60 6.53E-06 6.23E-06Attorney Docket No.2010363-0434 2 56.28% 948708 3.76 0.4831 0.4609 13364.18 5.26E-06 5.02E-06 3 56.27% 3489316 4.17 0.6477 0.6179 9965.78 1.05E-05 1.00E-06 4 40.05% 325956 4.72 0.4527 0.4318 14878.67 5.14E-06 4.91E-06 5 60.75% 92465646 4.18 0.5933 0.5660 12972.91 6.11E-06 5.82E-06Attorney Docket No.2010363-0434 Table 15. Effectiveness of LeMo-NaDe for generating optimal neural network models for ImageNET16-120 dataset using GPT-4 Turbo with different temperature values. TEM T = 0 T = 0 T = 0 T = 05 26.73% 832312 6.16 0.5988 0.5712 15291.25 5.07E-06 4.84E-06Attorney Docket No.2010363-0434 1 26.17% 4625464 6.74 0.6879 0.6562 16176.14 5.31E-06 5.06E-06 2 28.12% 418456 6.25 0.6454 0.6157 12878.29 5.47E-06 5.23E-06 T = 1 3 27.93% 111160 7.17 0.7335 0.6997 14035.10 5.23E-06 4.99E-06 4 10.13% 38024 8.92 0.8566 0.8173 17140.02 5.38E-06 5.13E-06 5 31.65% 3294584 7.16 0.7576 0.7227 15842.34 5.38E-06 5.14E-06Attorney Docket No.2010363-0434 Example 4: Illustrative results for the automated generation of a neural network architecture tailored to perform a particular task

[0162] This section provides an illustrative embodiment of a method for automated generation of a neural network architecture tailored to perform a particular task using Beam Search, following exemplary Algorithm 4. All experiments were run on a single NVIDIA A100 80GB GPU.

[0163] The beam search is used to track the optimal network (which may be obtained from LeMo-NADe or any other NNA) and modifying it for better performance. Here, every generated network was trained for 50 epochs with a batch size of 128, using Adam optimizer with initial learning rate of 0.001. The learning rate reduces by 75% of its initial value if validation loss doesn’t reduce for successive 5 epochs. To reduce overfitting, the training dataset was augmented with random rotation of 10 degrees, random horizontal flip, 10% random width shift, 10% random height shift, 10% random shear, and 10% random zoom. Finally, an optimal neural network was retrieved from LeMo-NADe and trained up to 500 epochs with a batch size of 512. Adam optimizer was used with initial learning rate of 0.01 and the learning rate reduced its initial values if validation loss did not reduce for successive 20 epochs. GPT-4o with temperature setting of 0.3 was used.

[0164] Six publicly available datasets were used: (1) MNIST, (2) Fashion MNIST, (3) Intel Scene, (4) EuroSAT, (5) Chest X-ray, and (6) Malaria. MNIST is large dataset used for hand written digit classification, which consist of 60k training images and 10k test images with resolution of 28x28. About 20% of images from training set were used as validation set. Fasion MNIST is another large dataset that contains 70k images of fashion items of ten different classes. Intel Scene dataset is published by Intel for image classification challenge. It consists of sixAttorney Docket No.2010363-0434 different scenes including forests, mountains, seas etc., associated with a scene recognition task. EuroSAT is a satellite image dataset that includes 27,000 annotated and geo-referenced samples. With a spatial resolution of 10 meters, the images cover ten different classes reflecting various land uses and land covers, including residential areas, forests, and water bodies. Chest X-ray dataset is the pneumonia classification dataset. It consists of total 5842 images of normal and pneumonia samples including 5218 training set and 624 images for test set. Finally, Malaria dataset consists of 27,558 cell images, with an equal number of parasitized and uninfected cells. It is utilized to classify malaria-infected cells and gives high-resolution microscopic images, which help to construct disease detection machine learning models. The datasets were resized into 64x64 size for experiments.

[0165] FIGs.14A-14B show accuracies for different datasets and configurations with beam widths of 3 and 5, respectively. The nomenclature in the charts follows the pattern V10W3, where V10 represents variant 10 (the number of trials to search for the optimal network) and W3 represents beam width 3. Similarly, V10W3_50 indicates variant 10 with beam width 3, with a 50% focus on inference time and 50% on accuracy. FIGs.14C-14D show inference time for different datasets and configurations with beam widths of 3 and 5, respectively. Tables 16-19 expand on the obtained results.

[0166] The accuracy plots show that the accuracy is almost similar for both 100% focus on accuracy and a 50 / 50 focus on inference time and accuracy in all cases. These graphs demonstrate that focusing on inference time during a search for new neural networks results in an efficient network that takes less time for inference while maintaining almost the same accuracy. This suggests that under the 50 / 50 setting, GPT-4o was able to find a lighter and shallower neural network architecture that performs equivalently well.Attorney Docket No.2010363-0434

[0167] An observation across the datasets is that the 50 / 50 policy either increases accuracy slightly, reduces the inference time, or achieves both. Here is a list of observations for 100 variants and beam width 3. MNIST: Table 16 shows that the best architecture for the MNIST dataset achieves an accuracy of 99.31% with V100W3 in 14.75 hours, while it reaches an accuracy of 99.22% with V100W3_50 in 13.75 hours, thereby saving 1 hour and still attaining a similar result. IntelScene: Accuracy is 88.47% with V100W3 in 12.69 hours, while it reaches 88.99% with V100W3_50 in 12.07 hours. Fashion MNIST: Accuracy is 93% with V100W3 in 14.33 hours, while it reaches 93.52% with V100W3_50 in 14.37 hours. EuroSAT: Accuracy is 97.30% with V100W3 in 20.34 hours, while it reaches 97.19% with V100W3_50 in 8.32 hours. Chest X-ray: Accuracy is 81.71% with V100W3 in 4.78 hours, while it reaches 82.05% with V100W3_50 in 4.52 hours. Malaria: Accuracy is 96.79% with V100W3 in 17.70 hours, while it reaches 96.83% with V100W3_50 in 16.25 hours.Attorney Docket No.2010363-0434 Table 16. Experimental Results with Beam Width of 3, Prioritizing Accuracy. Datas MNIntelS Fasi MNEuro Chest Mala. . . . . . . . . . . . Table 17. Experimental Results with Beam Width of 5, Prioritizing Accuracy. DatMNIntel Fas MNEuro ChestMalaria 96.52% 96.52% 3.35 96.59% 96.59% 8.89 96.89% 97.01% 15.86 96.97% 96.97% 18.69Attorney Docket No.2010363-0434 Table 18. Experimental Results with Beam Width of 3, Prioritizing 50% on accuracy and 50% on Inference time. Data MN IntelS Fas MNEuro Chest Malaa . . . . . . . . . . . . Table 19. Experimental Results with Beam Width of 5, Prioritizing 50% on accuracy and 50% on Inference time. Dat MN Intel Fas MNEuro ChestMalaria 96.95% 96.95% 3.43 96.97% 96.97% 5.68 97.08% 97.98% 6.88 96.61% 96.61% 12.46Attorney Docket No.2010363-0434 REFERENCES 1. Chua, M. et al. Tackling prediction uncertainty in machine learning for healthcare. Nat. Biomed. Eng.7, 711–718 (2023). 2. Bhardwaj, R. & Tripathi, I. An enhanced reversible data hiding algorithm using deep neural network for e-healthcare. J. Ambient Intell. Humaniz. Comput.14, 10567–10585 (2023). 3. Nandy, S. et al. An intelligent heart disease prediction system based on swarm-artificial neural network. Neural Comput. Appl.35, 14723–14737 (2023). 4. Jaafar, N. & Lachiri, Z. Multimodal fusion methods with deep neural networks and meta- information for aggression detection in surveillance. Expert. Syst. with Appl.211, 118523 (2023). 5. Mahum, R. et al. A robust framework to generate surveillance video summaries using combination of zernike moments and r-transform and deep neural network. Multimed. Tools Appl.82, 13811–13835 (2023). 6. Jan, Z. et al. Artificial intelligence for industry 4.0: Systematic review of applications, challenges, and opportunities. Expert. Syst. with Appl.216, 119456 (2023). 7. Raja Santhi, A. & Muthuswamy, P. Industry 5.0 or industry 4.0 s? introduction to industry 4.0 and a peek into the prospective industry 5.0 technologies. Int. J. on Interact. Des. Manuf. (IJIDeM) 17, 947–979 (2023). 8. Shafiq, M. et al. Continuous quality control evaluation during manufacturing using supervised learning algorithm for industry 4.0. The Int. J. Adv. Manuf. Technol.1–10 (2023).Attorney Docket No.2010363-0434 9. Rajput, D. S., Meena, G., Acharya, M. & Mohbey, K. K. Fault prediction using fuzzy convolution neural network on iot environment with heterogeneous sensing data fusion. Meas. Sensors 26, 100701 (2023). 10. Liyakat, K. K. S. Machine learning approach using artificial neural networks to detect malicious nodes in iot networks. In International Conference on Machine Learning, IoT and Big Data, 123–134 (Springer, 2023). 11. Thakkar, A. & Lohiya, R. Attack classification of imbalanced intrusion data for iot network using ensemble learning-based deep neural network. IEEE Internet Things J. (2023). 12. OpenAI, R. Gpt-4 technical report. arxiv 2303.08774. View Article 2, 13 (2023). 13. Wang, J. et al. El-nas: Efficient lightweight attention cross-domain architecture search for hyperspectral image classification. Remote. Sens.15, 4688 (2023). 14. Yang, T., He, Q. & Huang, L. Om-nas: pigmented skin lesion image classification based on a neural architecture search. Biomed. Opt. Express 14, 2153–2165 (2023). 15. Yang, Y., Wei, J., Yu, Z. & Zhang, R. A trustworthy neural architecture search framework for pneumonia image classification utilizing blockchain technology. The J. Supercomput.1–34 (2023). 16. Dong, P. et al. Rd-nas: Enhancing one-shot supernet ranking ability via ranking distillation from zero-cost proxies. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5 (IEEE, 2023).Attorney Docket No.2010363-0434 17. Wang, J. et al. Nas-dymc: Nas-based dynamic multi-scale convolutional neural network for sound event detection. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5 (IEEE, 2023). 18. Li, J. et al. Graph neural network architecture search for rotating machinery fault diagnosis based on reinforcement learning. Mech. Syst. Signal Process.202, 110701 (2023). 19. Yuan, W., Fu, C., Liu, R. & Fan, X. Ssob: searching a scene-oriented architecture for underwater object detection. The Vis. Comput.39, 5199–5208 (2023). 20. Jia, X. et al. Fast and accurate object detector for autonomous driving based on improved yolov5. Sci. reports 13, 1–13 (2023). 21. Mehta, R., Jurecková, O. & Stamp, M. A natural language processing approach to malware classification. J. Comput. Virol. Hacking Tech.1–12 (2023). 22. Girdhar, N., Coustaty, M. & Doucet, A. Benchmarking nas for article separation in historical newspapers. In International Conference on Asian Digital Libraries, 76–88 (Springer, 2023). 23. Real, E., Aggarwal, A., Huang, Y. & Le, Q. V. Regularized evolution for image classifier architecture search. In Proceedings of the AAAI conference on artificial intelligence, vol.33, 4780–4789 (2019). 24. Liu, C. et al. Progressive neural architecture search. In Proceedings of the European conference on computer vision (ECCV), 19–34 (2018). 25. Cai, H., Chen, T., Zhang, W., Yu, Y. & Wang, J. Efficient architecture search by network transformation. In Proceedings of the AAAI Conference on Artificial Intelligence, vol.32 (2018).Attorney Docket No.2010363-0434 26. Pham, H., Guan, M., Zoph, B., Le, Q. & Dean, J. Efficient neural architecture search via parameters sharing. In International conference on machine learning, 4095–4104 (PMLR, 2018). 27. Liu, H., Simonyan, K. & Yang, Y. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055 (2018). 28. Ying, C. et al. Nas-bench-101: Towards reproducible neural architecture search. In International conference on machine learning, 7105–7114 (PMLR, 2019). 29. Dong, X. & Yang, Y. Nas-bench-201: Extending the scope of reproducible neural architecture search. arXiv preprint arXiv:2001.00326 (2020). 30. Krizhevsky, A. & Hinton, G. Learning multiple layers of features from tiny images. Tech. Rep., Toronto, ON, Canada (2009). 31. Chrabaszcz, P., Loshchilov, I. & Hutter, F. A downsampled variant of imagenet as an alternative to the cifar datasets. arXiv preprint arXiv:1707.08819 (2017). 32. Ye, P. et al. β-darts: Beta-decay regularization for differentiable architecture search. In 2022 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10864–10873 (IEEE, 2022). 33. Movahedi, S. et al. λ -darts: Mitigating performance collapse by harmonizing operation selection among cells. arXiv preprint arXiv:2210.07998 (2022). 34. Zheng, M. et al. Can gpt-4 perform neural architecture search? arXiv preprint arXiv:2304.10970 (2023). 35. Achiam, J. et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023).Attorney Docket No.2010363-0434 36. Wang, H. et al. Graph neural architecture search with gpt-4. arXiv preprint arXiv:2310.01436 (2023). 37. Helber, P., Bischke, B., Dengel, A. & Borth, D. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. (2019). 38. Rajaraman, S. et al. Pre-trained convolutional neural networks as feature extractors toward improved malaria parasite detection in thin blood smear images. PeerJ 6, e4568 (2018). 40. Loni, M., Sinaei, S., Zoljodi, A., Daneshtalab, M. & Sjödin, M. Deepmaker: A multi- objective optimization framework for deep neural networks in embedded systems. Microprocess. Microsystems 73, 102989 (2020). 41. Suganuma, M., Kobayashi, M., Shirakawa, S. & Nagao, T. Evolution of deep convolutional neural networks using cartesian genetic programming. Evol. computation 28, 141–163 (2020). 42. Ren, J. et al. Eigen: Ecologically-inspired genetic approach for neural network structure searching from scratch. In Proceedings of the IEEE / CVF conference on computer vision and pattern recognition, 9059–9068 (2019). 43. Xie, L. & Yuille, A. Genetic cnn. In Proceedings of the IEEE international conference on computer vision, 1379–1388 (2017). 44. Lu, Z. et al. Multi-criterion evolutionary design of deep convolutional neural networks. arXiv preprint arXiv:1912.01369172 (2019).Attorney Docket No.2010363-0434 45. Kandasamy, K., Neiswanger, W., Schneider, J., Poczos, B. & Xing, E. P. Neural architecture search with Bayesian optimisation and optimal transport. Adv. neural information processing systems 31 (2018). 46. Elsken, T., Metzen, J.-H. & Hutter, F. Simple and efficient architecture search for convolutional neural networks. arXiv preprint arXiv:1711.04528 (2017). 47. Dong, X. & Yang, Y. Searching for a robust neural architecture in four gpu hours. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 1761– 1770 (2019). 48. Chu, X. et al. Darts-: robustly stepping out of performance collapse without indicators. arXiv preprint arXiv:2009.01027 (2020). 49. Chen, X., Wang, R., Cheng, M., Tang, X. & Hsieh, C.-J. Drnas: Dirichlet neural architecture search. arXiv preprint arXiv:2006.10355 (2020). 50. Hu, Y., Wang, X., Li, L. & Gu, Q. Improving one-shot nas with shrinking-and-expanding supernet. Pattern Recognit.118, 108025 (2021). 51. Chu, X., Zhang, B. & Xu, R. Fairnas: Rethinking evaluation fairness of weight sharing neural architecture search. In Proceedings of the IEEE / CVF International Conference on computer vision, 12239–12248 (2021). 52. Xiao, H., Wang, Z., Zhu, Z., Zhou, J. & Lu, J. Shapley-nas: Discovering operation contribution for neural architecture search. In Proceedings of the IEEE / CVF conference on computer vision and pattern recognition, 11892–11901 (2022).Attorney Docket No.2010363-0434 53. Yu, K., Ranftl, R. & Salzmann, M. Landmark regularization: Ranking guided super-net training in neural architecture search. In Proceedings of the IEEE / CVF Conference on computer vision and pattern recognition, 13723–13732 (2021). 54. Cavagnero, N., Robbiano, L., Caputo, B. & Averta, G. Freerea: Training-free evolution-based architecture search. In Proceedings of the IEEE / CVF Winter Conference on Applications of Computer Vision, 1493–1502 (2023). 55. Zheng, X. et al. Neural architecture search with representation mutual information. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 11912– 11921 (2022). 56. Strubell, E., Ganesh, A. & McCallum, A. Energy and policy considerations for deep learning in nlp. arXiv preprint arXiv:1906.02243 (2019). 57. Shoukang Hu, Sirui Xie, Hehui Zheng, Chunxiao Liu, Jianping Shi, Xunying Liu, and Dahua Lin. Dsnas: Direct neural architecture search without parameter retraining. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pages 12084–12092, 2020. 58. Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Guo-Jun Qi, Qi Tian, and Hongkai Xiong. Pc-darts: Partial channel connections for memory-efficient architecture search. arXiv preprint arXiv:1907.05737, 2019. 59. Miao Zhang, Steven W Su, Shirui Pan, Xiaojun Chang, Ehsan M Abbasnejad, and Reza Haffari. idarts: Differentiable architecture search with stochastic implicit gradients. In International Conference on Machine Learning, pages 12557–12566. PMLR, 2021.Attorney Docket No.2010363-0434 60. Xuanyi Dong and Yi Yang. Searching for a robust neural architecture in four gpu hours. In Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, pages 1761–1770, 2019. 61. L. Deng, “The mnist database of handwritten digit images for machine learning research,” IEEE signal processing magazine, vol.29, no.6, pp.141–142, 2012. 62. H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747, 2017. 63. Intel, “Intel image classification challenge.” https: / / www.kaggle.com / datasets / puneet6060 / intel-image-classification. Kaggle Dataset. 64. D. Kermany, K. Zhang, and M. Goldbaum, “Labeled optical coherence tomography (oct) and chest x-ray images for classification,” Mendeley Data, v2. EQUIVALENTS

[0168] Elements of different implementations described herein may be combined to form other implementations not specifically set forth above. Elements may be left out of the processes, computer programs, databases, etc. described herein without adversely affecting their operation. In addition, the logic flows depicted in the figures do not require the particular order shown, or sequential order, to achieve desirable results. Various separate elements may be combined into one or more individual elements to perform the functions described herein.

[0169] In some implementations, various modules described herein can be separated, combined or incorporated into single or combined modules. Modules depicted in the figures are not intended to limit the systems described herein to the software architectures shown therein.Attorney Docket No.2010363-0434

[0170] Throughout the description, where apparatus and systems are described as having, including, or comprising specific components, or where processes and methods are described as having, including, or comprising specific steps, it is contemplated that, additionally, there are apparatus, and systems of the present invention that consist essentially of, or consist of, the recited components, and that there are processes and methods according to the present invention that consist essentially of, or consist of, the recited processing steps.

[0171] It should be understood that the order of steps or order for performing certain action is immaterial so long as the invention remains operable. Moreover, two or more steps or actions may be conducted simultaneously.

[0172] While the invention has been particularly shown and described with reference to specific preferred embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

Attorney Docket No.2010363-0434 What is claimed is:

1. A method of automatically generating a neural network architecture (NNA) tailored for a given task, the method comprising: (a) receiving, by a processor of a computing device, user input comprising (i) a task description, (ii) one or more dataset(s) associated with the given task, and (iii) one or more input metrics to guide the automatic NNA generation process; (b) automatically generating, by the processor, a command based at least on the received user input; (c) transmitting, by the processor, the automatically generated command to a Large Language Model (LLM) and / or an Artificial General Intelligence (AGI) model (an LLM / AGI) and receiving from the LLM / AGI, in response to said command, a first generated candidate neural network architecture (a first candidate NNA); (d) evaluating, by the processor, the first candidate NNA by at least partially training the first candidate NNA and validating the first candidate NNA using a subset of the user input, thereby determining values of one or more evaluation based metrics associated with the first candidate NNA; (e) repeating steps (b) to (d) one or more times to generate, by the processor, a second and / or subsequent candidate NNA(s) and to determine associated evaluation based metrics for each candidate NNA; and (f) identifying, by the processor, a generated candidate NNA as satisfactory for the given task upon attaining a terminating condition.Attorney Docket No.2010363-0434 2. The method of claim 1, wherein step (b) comprises automatically performing, by the processor, a procedure to resolve any conflict in the automatically generated command.

3. The method of claim 1 or 2, wherein the one or more evaluation based metrics associated with a candidate NNA comprise one or more members selected from the group consisting of: (i) a training accuracy of the candidate NNA, (ii) a validation accuracy of the candidate NNA, (iii) an energy required for evaluating the training set, (iv) an energy required for evaluating the validation set, (v) a value of frames per second (FPS) of the candidate NNA, and (vi) one or more parameters of the candidate NNA.

4. The method of any one of the preceding claims, wherein the one or more input metrics to guide the automatic NNA generation process comprise one or more members selected from the group consisting of: (i) a weight for accuracy, (ii) a weight for FPS, and (iii) a weight for energy.

5. The method of any one of the preceding claims, wherein the one or more input metrics to guide the automatic NNA generation process comprise internal metrics associated with internal settings of the LLM / AGI.

6. The method of any one of the preceding claims, wherein the one or more input metrics to guide the automatic NNA generation process comprise one or more members selected from the group consisting of: (i) a priority of training accuracy, (ii) a threshold of training accuracy, (iii) a priority of validation accuracy, (iv) a threshold of validation accuracy, (v) a priority of energy required for evaluating the training set, (vi) a threshold of energy required for evaluating theAttorney Docket No.2010363-0434 training set, (vii) a priority of energy required for evaluating the validation set, (viii) a threshold of energy required for evaluating the validation set, (ix) a priority of FPS, (x) a threshold of FPS, (xi) a priority of each of one or more parameters of a candidate NNA, (xii) a threshold of each of one or more parameters of a candidate NNA, (xiii) a threshold value to check overfitting, (xiv) a threshold value to check underfitting.

7. The method of any one of the preceding claims, wherein the command generated in step (b) comprises a set of architectural commands, and wherein each architectural command in the set of architectural commands comprises: (i) a type of neural network architectural change; and (ii) a weight associated with prioritization of the architectural command.

8. The method of claim 7, wherein the procedure to resolve any conflict in the automatically generated command comprises setting a lower weight in a pair of conflicting architectural commands as zero.

9. The method of any one of the preceding claims, wherein the one or more input metrics to guide the automatic NNA generation process are updated in step (e) based on the one or more evaluation based metrics.

10. The method of claim 9, wherein the one or more input metrics to guide the automatic NNA generation process comprise the threshold of training accuracy, and wherein, if the training accuracy of the candidate NNA does not meet the threshold of training accuracy, weights of one or more architectural commands are increased.Attorney Docket No.2010363-0434 11. The method of claim 9, wherein the one or more input metrics to guide the automatic NNA generation process comprise the threshold of validation accuracy, and wherein, if the validation accuracy of the candidate NNA does not meet the threshold of validation accuracy, weights of one or more architectural commands are increased.

12. The method of claim 9, wherein the one or more input metrics to guide the automatic NNA generation process comprise the threshold of energy required for evaluating the training set, and wherein, if the energy required for evaluating the training set satisfies the threshold of energy required for evaluating the training set, weights of one or more architectural commands are increased.

13. The method of claim 9, wherein the one or more input metrics to guide the automatic NNA generation process comprise the threshold of energy required for evaluating the validation set, and wherein, if the energy required for evaluating the validation set satisfies the threshold of energy required for evaluating the validation set, weights of one or more architectural commands are increased.

14. The method of claim 9, wherein the one or more input metrics to guide the automatic NNA generation process comprise the threshold of FPS, andAttorney Docket No.2010363-0434 wherein, if the value of FPS does not meet the threshold of FPS, weights of one or more architectural commands are increased.

15. The method of claim 9, wherein the one or more input metrics to guide the automatic NNA generation process comprise the threshold value to check overfitting, and wherein, if a difference between the training accuracy and the validation accuracy satisfies the threshold value to check overfitting, one or more weights of architectural commands are increased.

16. The method of claim 9, wherein the one or more input metrics to guide the automatic NNA generation process comprise the threshold value to check underfitting, and wherein, if a difference between the validation accuracy and the training accuracy satisfies the threshold value to check underfitting, one or more weights of architectural commands are increased.

17. The method of any one of the preceding claims, wherein the user input in step (a) comprises a task description, and wherein the task description is associated with image recognition.

18. The method of claim 17, wherein the image recognition is associated with one or more objects, said one or more objects selected from the group consisting of: alphanumeric symbols, cloth, geographical features, features in satellite images, features of the human body and associated conditions, and cellular features.Attorney Docket No.2010363-0434 19. A method of automatically generating a neural network architecture (NNA) tailored for a given task, the method comprising: (a) receiving, by a processor of a computing device, user input comprising (i) a task description, (ii) one or more dataset(s) associated with the given task, and (iii) one or more input metrics to guide the automatic NNA generation process; (b) automatically generating, by the processor, a command based at least on the received user input; (c) transmitting, by the processor, the automatically generated command to a Large Language Model (LLM) and / or an Artificial General Intelligence (AGI) model (an LLM / AGI) and receiving from the LLM / AGI, in response to said command, the NNA.

20. The method of claim 19, wherein the generated command in step (b) is based on (i) the user input, (ii) a set of rules, and, optionally, (iii) one or more previously predicted models for the given task and / or its / their corresponding evaluation reports.

21. A method of automatically generating a neural network architecture (NNA) tailored for a given task, the method comprising: (a) receiving, by a processor of a computing device, user input comprising (i) a task description, (ii) one or more dataset(s) associated with the given task, (iii) one or more input metrics to guide the automatic NNA generation process, and, optionally, (iv) an initial NNA; (b) automatically generating, by the processor, a command based at least on the received user input;Attorney Docket No.2010363-0434 (c) transmitting, by the processor, the automatically generated command to a Large Language Model (LLM) and / or an Artificial General Intelligence (AGI) model (an LLM / AGI) and receiving from the LLM / AGI, in response to said command, a first generated candidate neural network architecture (a first candidate NNA); (d) evaluating, by the processor, the first candidate NNA by at least partially training the first candidate NNA and validating the first candidate NNA using a subset of the user input, thereby determining values of one or more evaluation based metrics associated with the first candidate NNA; (e) repeating steps (b) to (d) one or more times to generate, by the processor, a second and / or subsequent candidate NNA(s) and to determine associated one or more evaluation based metrics for each candidate NNA; and (f) identifying, by the processor, a generated candidate NNA as satisfactory for the given task upon attaining a terminating condition.

22. The method of claim 21, comprising: (g) filtering, by the processor, the candidate NNA(s) by: (1) sorting the candidate NNA(s) according to associated one or more evaluation based metrics into a search tree and (2) generating selected candidate NNA(s) using the search tree according to a search algorithm.

23. The method of claim 22, wherein the search algorithm comprises a beam search with a beam width.

24. The method of claim 22 or 23, comprising:Attorney Docket No.2010363-0434 (h) repeating steps (b) to (g) one or more times to generate, by the processor, revised candidate NNA(s) and to determine associated one or more evaluation based metrics for each revised candidate NNA, wherein, at step (b), the user input comprises one of the selected candidate NNA(s), wherein, at step (b), the one or more input metrics to guide the automatic NNA generation process are updated based on the one or more evaluation based metrics, and wherein, at step (g), resulting generated candidate NNA(s) and associated one or more evaluation based metrics are appended to the search tree.

25. The method of claim 24, wherein the one of the selected candidate NNA(s) is selected from the search tree in a sequential order.

26. The method of claim 24, wherein the one of the selected candidate NNA(s) is selected from the search tree using an algorithm.

27. The method of any one of claims 21-26, wherein one or more evaluation based metrics comprise an inference accuracy of the candidate NNA and / or an inference time of the candidate NNA.

28. The method of any one of claims 21-27, wherein the one or more input metrics to guide the automatic NNA generation process comprise a weight of accuracy and / or a weight of inference time.Attorney Docket No.2010363-0434 29. The method of any one of claims 21-26, wherein the one or more evaluation based metrics associated with a candidate NNA comprise one or more members selected from the group consisting of: (i) a training accuracy of the candidate NNA, (ii) a validation accuracy of the candidate NNA, (iii) an energy required for evaluating the training set, (iv) an energy required for evaluating the validation set, (v) a value of frames per second (FPS) of the candidate NNA, and (vi) one or more parameters of the candidate NNA.

30. The method of any one of claims 21-26 and 29, wherein the one or more input metrics to guide the automatic NNA generation process comprise one or more members selected from the group consisting of: (i) a weight for accuracy, (ii) a weight for FPS, and (iii) a weight for energy.

31. A system comprising a processor of a computing device and a memory having instructions stored thereon, which, when executed by the processor, cause the processor to perform the method of any one of the preceding claims.

Images