Dynamic Neural Network Model Selection for Image Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices face challenges in performing high-performance image processing due to insufficient hardware resources, complex pre-processing requirements, and difficulties in upgrading hardware and software.
Innovation Solution
An electronic device that includes memory storing neural network models and instructions to identify scene types, transmit this information to a server, receive and apply corresponding neural network models and parameters for image processing, and update models based on user input and scene type changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If high-performance neural network models are used for image processing, then image processing performance is improved, but hardware resources (processing capacity and storage capacity) become insufficient
Solution Approach 1:
The patent segments the neural network model into multiple components: a first neural network model for scene type identification and a second neural network model for image processing. This segmentation allows the device to use a lightweight first model for classification and only invoke the heavier second model when needed, reducing overall hardware resource consumption while maintaining image processing performance.
Solution Approach 2:
The patent introduces a scene type identification mechanism as an intermediary between the input image and the main image processing neural network model. This intermediary first model classifies the input scene type and retrieves corresponding processing parameters from a database, avoiding the need to always use the full high-performance model and thus conserving hardware resources.
2Productivity
If advanced neural network models are deployed, then image processing capability is improved, but pre-processing complexity increases
Solution Approach 1:
The patent divides the pre-processing stage into two segments: a simple first neural network model for scene type classification and a parameter retrieval step from database. This segmentation simplifies the pre-processing complexity compared to using a single complex model, as the first model is lightweight and the parameter lookup is straightforward.
Solution Approach 2:
The patent performs preliminary scene type identification using the first neural network model before invoking the main image processing model. This preliminary action prepares the system by determining the appropriate processing parameters in advance, simplifying the overall pre-processing workflow and reducing the complexity of the main model's input requirements.
3Productivity
If high-performance models are used, then image processing quality is improved, but hardware upgrade difficulty increases
Solution Approach 1:
The patent implements a dynamic model selection mechanism where the system can switch between a lightweight first neural network model and a more capable second neural network model based on the scene type and processing requirements. This dynamic approach allows the device to achieve high processing quality when needed while maintaining ease of operation with the simpler first model for routine tasks, facilitating easier hardware deployment and upgrades.
Solution Approach 2:
The patent changes the operational parameters of the neural network system by using different models for different scene types. The first model handles general scene classification with fewer parameters, while the second model handles specific image processing tasks with appropriate parameters retrieved from the database. This parameter-based adaptation allows high processing quality without requiring the device to always maintain the full computational capacity of the second model.
Data Source
AI summary
An electronic device is configured to identify a scene type of a first frame included in a content by inputting the first frame into the first neural network model, transmit the scene type of the first frame to a server, receive, from the server in response to the transmitted scene type of the first frame, a second neural network model and a first parameter corresponding to the scene type of the first frame, replace the first neural network model with the second neural network model, and perform image processing on the first frame based on the first parameter, and in which the second neural network model is one of a plurality of second neural network models respectively corresponding to a plurality of scene types that can be output from the first neural network model.