Proxy Evaluation Feedback for Neural Architecture Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional machine learning (ML) model design for deep neural networks (DNNs) is time-consuming and resource-intensive, with proxy tasks often resulting in less accurate models, and Neural Architecture Search (NAS) methods can be computationally expensive, making AI/ML techniques inaccessible to users without large compute resources.
Innovation Solution
A system that uses proxy evaluation feedback with semi-supervised learning (SSL) mechanisms to provide near real-time performance estimates for DNN models, integrating proxy scoring functions and objective mappings in a hardware-aware manner, reducing the time and computational resources required for ML model engineering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If full training and validation cycles are used for DNN model design, then model accuracy is improved, but time consumption and computational resource requirements increase significantly
Solution Approach 1:
The patent applies preliminary action by training a proxy model beforehand to predict performance metrics. Instead of performing full training cycles for every architecture evaluation, the system uses the pre-trained proxy model to provide quick predictions during the architecture search process, significantly reducing the time required while maintaining reasonable accuracy estimates.
Solution Approach 2:
The patent introduces a proxy model as an intermediary between the neural network architecture and the performance evaluation. This proxy model acts as a mediator that translates architectural specifications into predicted performance metrics without requiring full training, thus bridging the gap between rapid evaluation and accurate assessment.
2Loss of time
If proxy tasks are used to reduce training time, then time consumption is reduced, but model accuracy decreases
Solution Approach 1:
The patent creates a simplified copy of the training process through the proxy model. This proxy model is trained on a subset of data and uses fewer epochs, creating a lightweight version that mimics the behavior of full training. The copy provides sufficiently accurate predictions for architecture comparison without incurring the full computational cost of complete training cycles.
3Productivity
If Neural Architecture Search algorithms are used to automatically discover ideal models, then model design efficiency is improved, but computational resource requirements increase
Solution Approach 1:
The patent implements feedback mechanisms where the proxy model's predictions are continuously refined based on actual training results. The system uses feedback loops to update the proxy model with information from full training runs, improving its accuracy over time while maintaining the efficiency gains of using proxy evaluations for the majority of architecture searches.
4Productivity
If proxy functions are used to predict model accuracy, then evaluation speed is improved, but prediction accuracy correlates poorly with actual performance
Solution Approach 1:
The patent applies parameter changes by adjusting the training regime of the proxy model to optimize the balance between speed and accuracy. The system experiments with different training parameters such as batch size, learning rate, and number of epochs to find the optimal configuration that provides sufficiently accurate predictions while maintaining fast evaluation speeds.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present disclosure is related to artificial intelligence (AI), machine learning (ML), and Neural Architecture Search (NAS) technologies, and in particular, to Deep Neural Network (DNN) model engineering techniques that use proxy evaluation feedback. The DNN model engineering techniques discussed herein provide near real-time feedback on model performance via low-cost proxy scores without requiring continual training and/or validation cycles, iterations, epochs, etc. In conjunction with the proxy-based scoring, semi-supervised learning mechanisms are used to map proxy scores to various model performance metrics. Other embodiments may be described and/or claimed.