Low-Resolution Image Restoration With Text-Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing technologies struggle to effectively restore low-resolution images by enhancing the visibility of distorted subjects, such as license plates, in real-time applications using artificial intelligence.
Innovation Solution
An image restoration model is trained using a teacher model through knowledge distillation to enhance low-resolution images by combining text probability maps and feature information, utilizing an encoder to extract features and a decoder to generate high-resolution outputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If an image restoration model is trained using knowledge distillation from a teacher model, then the visibility and clarity of distorted subjects in low-resolution images is improved, but the device complexity and training requirements increase
Solution Approach 1:
The patent applies knowledge distillation by training a student model (image restoration model) to copy the learned features and patterns from a teacher model that processes high-resolution images. The student model learns to extract meaningful features and restore image quality by mimicking the teacher model's decision-making process, thereby achieving high restoration accuracy without requiring the same computational resources as the teacher model.
Solution Approach 2:
The patent introduces an intermediary training mechanism where a teacher model first processes high-resolution images to generate feature maps and probability maps, which then serve as training data for the student model. This intermediary approach allows the student model to learn from high-resolution data indirectly, reducing the direct computational burden while maintaining restoration quality.
2Measurement precision
If the model combines text probability maps and feature information through fusion layers, then character recognition accuracy is improved, but the processing time and computational resources increase
Solution Approach 1:
The patent segments the image processing task into distinct functional components: an encoder for extracting visual features, a sub-model for generating text probability maps, and a fusion layer for combining these inputs. This segmentation allows each component to be optimized independently and enables parallel processing, reducing overall computation time while maintaining recognition accuracy.
Solution Approach 2:
The patent performs preliminary extraction of feature information and generation of text probability maps before the final fusion and restoration steps. By preparing these intermediate representations in advance, the model reduces the computational burden during the critical fusion phase, thereby decreasing processing time while maintaining accuracy.
Data Source
AI summary
According to an embodiment, an electronic device performs, by using an input image with a first resolution and a ground truth image with a second resolution greater than the first resolution, training of an image restoration model including a sub-model trained to output a text probability map indicating one or more characters associated with the input image, an encoder to extract feature information from the input image, a fusion layer to combine the text probability map and the feature information, and a decoder to generate an output image with the second resolution, that is connected to the fusion layer. The electronic device provides the image restoration model as a portion of a software application to restore an image. The electronic device trains the encoder using feature information generated by a teacher model that is used to train the sub-model based on knowledge distillation.