Motion Vector Confidence Scoring via Neural Network Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video encoding methods face challenges in accurately determining the confidence of motion vectors, leading to ambiguous conclusions and erroneous matches, especially in low contrast and high noise scenarios, which affects the accuracy of motion analysis in applications like self-driving cars and computer vision.
Innovation Solution
The proposed solution involves generating confidence scores for motion vectors using a neural network that combines spatial, temporal, and virtual feature data to classify the likelihood of a motion vector representing the true object motion, rejecting outlier vectors, and providing a trustworthiness score for motion analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If block matching estimation is performed to determine motion vectors between video frames, then motion analysis capability is enabled, but accuracy deteriorates in low contrast and high noise scenarios leading to erroneous matches
Solution Approach 1:
The patent introduces an intermediary confidence determination process that acts as a mediator between block matching estimation and motion analysis. This intermediary layer evaluates multiple features (spatial, temporal, virtual) to assess the reliability of motion vectors before they are used in downstream applications, thereby resolving the contradiction by filtering out erroneous matches without reducing the fundamental block matching precision
Solution Approach 2:
The patent changes the parameter space by introducing confidence scores as an additional evaluation dimension. Instead of relying solely on traditional block matching metrics, the system evaluates motion vectors through multiple parameter changes including spatial features, temporal features, and virtual features, allowing the system to distinguish between reliable and unreliable motion vectors in challenging scenarios
2Reliability
If confidence determination using neural networks is implemented, then motion vector reliability is improved, but computational complexity increases
Solution Approach 1:
The patent segments the confidence determination process into distinct feature extraction stages (spatial features, temporal features, virtual features) that can be processed independently. This segmentation allows the neural network to handle complex reliability assessment through modular feature processing, reducing overall computational complexity while maintaining high reliability
Solution Approach 2:
The patent performs preliminary feature extraction and evaluation before the main confidence determination. By pre-computing spatial, temporal, and virtual features and organizing them into structured formats, the system reduces the computational burden on the neural network during actual motion vector evaluation, thereby managing complexity while preserving reliability
3Measurement precision
If multiple feature types are combined for confidence scoring, then classification accuracy is improved, but data processing requirements increase
Solution Approach 1:
The patent merges multiple feature types (spatial, temporal, virtual) into a unified confidence scoring framework. By combining these diverse feature sources through the neural network, the system achieves higher classification accuracy for motion vector reliability while managing data processing requirements through integrated feature processing
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems, apparatus, articles of manufacture, and methods are disclosed to determine confidence of motion vectors. Examples disclosed herein are to generate feature data associated with a motion vector, the motion vector generated based on a first block of pixel data in a first video frame and a second block of pixel data in a second video frame, determine a confidence score for the motion vector based on a model and the feature data, and concatenate the motion vector and the confidence score to output an estimated likelihood that the motion vector is accurate.