Visual-Language Model Encoder for Image Quality Assessment

The patent uses a visual-language model (VLM) to generate synthetic training data that replicates real image-quality assessment scenarios. The VLM creates simulated image pairs with ground truth quality labels by applying degradation models and generating corresponding descriptive text, effectively copying real data characteristics without requiring additional real captured data. This resolves the contradiction by expanding the quantity of training data while maintaining data quality and diversity.

The patent introduces a VLM as an intermediary system between the limited real image data and the neural network trainer. The VLM acts as a mediator that generates synthetic training examples by taking real images, applying degradation effects, and creating corresponding quality labels and descriptions. This intermediary enables the neural network to train on expanded synthetic data while preserving the underlying patterns from real data.

The patent segments the system into distinct functional components: a pre-existing VLM for data generation, a target encoder for image feature extraction, and a fine-tuning module for optimization. The VLM handles data synthesis while the encoder focuses on visual processing, separating complex functions into independent modules. This segmentation allows the system to leverage the VLM's data generation capability without requiring a complete redesign of the entire architecture.

The patent performs preliminary actions by pre-training the VLM and pre-processing real images through degradation models before the main training process. The VLM is pre-configured with language and vision capabilities, and real images are pre-enhanced with various degradation effects to create a diverse training corpus. This preliminary preparation reduces the complexity of the main training process by having ready-to-use synthetic data and pre-processed features.