Neural Network Image Processing Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing technologies face challenges in improving image quality, especially for videos, due to differences in image quality characteristics between training images and input images, leading to performance deterioration and flicker distortion.
Innovation Solution
An image processing device and method that utilize a neural network model to adapt to the quality of a first image by generating training data based on the image quality and viewing information, and then train a meta model to perform image quality processing, thereby enhancing image quality without requiring extensive retraining for each image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a neural network model is trained using traditional methods with fixed training data, then the model can achieve good performance on similar images, but it performs poorly when image quality characteristics differ from training data
Solution Approach 1:
The patent implements dynamic adaptability by enabling the neural network model to be retrained on-device with newly acquired training data from actual input images. This allows the model to dynamically adjust to different image quality characteristics rather than being static after initial training, resolving the contradiction between reliability on training data and adaptability to new data distributions
Solution Approach 2:
The system performs self-service by conducting on-device learning where the neural network model retrains itself using training data generated from actual input images. This self-retraining capability allows the model to automatically adapt to different image quality characteristics without requiring external retraining, simultaneously improving reliability and adaptability
2Adaptability or versatility
If on-device learning is implemented to adapt the model to input data, then adaptability improves, but learning complexity and computation time increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-training the neural network model offline before deployment on the device. This pre-trained model serves as a foundation that requires minimal further training when deployed, significantly reducing the learning complexity and computation time required for on-device adaptation while maintaining high adaptability
Solution Approach 2:
The system changes parameters by using a pre-trained model with optimized architecture and parameters, then performing lightweight fine-tuning on-device rather than full retraining. This parameter adjustment approach reduces computation complexity while preserving adaptability to different image characteristics
3Manufacturing precision
If independent models are trained for each image to achieve high quality restoration, then image quality improves, but temporal consistency deteriorates causing flicker distortion in videos
Solution Approach 1:
The patent merges multiple training approaches by combining the benefits of image-specific adaptation with temporal consistency constraints. The model integrates information across multiple frames while performing adaptation, unified by a loss function that simultaneously optimizes for restoration quality and temporal stability, resolving the contradiction between precision and consistency
4Manufacturing precision
If training data is constructed by referring to input images with repetitive patterns, then performance improves for such images, but performance deteriorates for images lacking repeated components
Solution Approach 1:
The system uses self-service through on-device learning where the model generates its own training data from actual input images regardless of whether they contain repetitive patterns. This approach allows the model to learn from diverse image types including those without repeated components, improving both restoration quality and adaptability across different image categories
Data Source
AI summary
Provided are an image processing device and an operating method of the same. The image processing device includes a memory storing one or more instructions, and at least one processor processing circuitry, and memory storing one or more instructions that, when executed by the at least one processor individually or collectively, cause the image processing device to obtain a neural network model corresponding to a quality of an input image and viewing information related to the input image. The at least one processor is configured to generate training data, based on the quality of the input image and the viewing information. The at least one processor is configured to train the neural network model by using the training data. The at least one processor is configured to obtain an image quality processed output image from the input image, based on the trained neural network model.