Video Quality Assessment Model Training With 3D CNN and Attention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video quality assessment methods, such as PSNR, SSIM, and VMAF, suffer from incomplete feature extraction and ambiguous boundary distinction, leading to inaccurate video quality assessments.
Innovation Solution
A model training method involving an initial video quality assessment model trained with reference and distorted video data, using a three-dimensional convolutional neural network for motion information extraction, combined with an attention model and data fusion, to achieve complete feature extraction and accurate boundary detection, ensuring independence and diversity of training data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional video quality assessment methods (PSNR, SSIM, VMAF) are used, then the assessment process is simple and fast, but the feature extraction is incomplete and boundary distinction is ambiguous, leading to inaccurate assessment results
Solution Approach 1:
The patent divides the video quality assessment model into multiple specialized components: a three-dimensional convolutional neural network for motion information extraction, an attention model for feature weighting, and a data fusion module for integrating multiple features. This segmentation allows each component to focus on specific aspects of video quality, improving overall assessment accuracy while managing complexity through modular design
Solution Approach 2:
The patent introduces three-dimensional convolutional operations to process video data in temporal and spatial dimensions simultaneously, adding a time dimension to the traditional two-dimensional spatial processing. This enables the model to capture motion information and temporal correlations, significantly improving feature extraction capability and boundary distinction in video quality assessment
2Measurement precision
If deep learning models with complex structures are used to improve feature extraction, then video quality assessment accuracy improves, but the risk of overfitting increases and generalization capability deteriorates
Solution Approach 1:
The patent implements comprehensive data preprocessing and augmentation techniques before model training, including video data normalization, distortion type classification, and diverse training data preparation. This preliminary action ensures the model learns from high-quality, well-structured data, improving generalization capability while maintaining feature extraction effectiveness
Solution Approach 2:
The patent employs a feedback mechanism where the model's assessment results are continuously evaluated against ground truth MOS values, and the training process is adjusted based on performance metrics. This includes validation set monitoring, loss function optimization, and iterative model refinement, which prevents overfitting by providing continuous feedback on generalization performance
3Adaptability or versatility
If comprehensive training data with multiple distortion types is used, then the model's adaptability to different video conditions improves, but the data processing complexity and training time increase
Solution Approach 1:
The patent performs preliminary classification and organization of training data by distortion type, severity level, and video content category before training. This preprocessing creates structured datasets that can be efficiently sampled and processed, allowing the model to learn diverse distortion patterns without excessive training time
Solution Approach 2:
The patent implements periodic training cycles with varying data sampling strategies, where different distortion types and video sequences are rotated through the training set in structured patterns. This periodic action ensures comprehensive coverage of all distortion types while maintaining efficient training throughput through systematic data utilization
Data Source
Figure 1~2
Figure 3~5
Figure 6~7
AI summary
Provided in the present disclosure is a model training method for video quality assessment. The method comprises: acquiring training video data, wherein the training video data comprises reference video data and distortion video data; determining a mean opinion score (MOS) value of the training video data; and training a preset initial video quality assessment model according to the training video data and the MOS value thereof, until a convergence condition is reached, so as to obtain a final video quality assessment model. Further provided in the present disclosure are a video quality assessment method and apparatus, and a device and a medium.