GAN Optical Flow Determination via Distribution-Based Image Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Accurate and effective calculation of optical flow in image analysis is challenging due to the time-consuming task of finding pixel correspondences between consecutive frames, often with no exact pixel intensity correspondence or unique solutions.
Innovation Solution
A generative adversarial network (GAN) system is employed, comprising a generator sub-network that creates distribution-based images representing optical flow and a discriminator sub-network that determines the accuracy of these images, allowing for unsupervised learning of optical flow without ground truth, using artificial neural networks to estimate optical flow between consecutive video frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional methods are used to calculate optical flow by finding pixel correspondences between consecutive frames, then measurement precision can be achieved, but the calculation becomes time-consuming and computationally intensive
Solution Approach 1:
The patent replaces traditional mechanical/image processing methods with a neural network-based system. The neural network learns optical flow patterns from training data and directly predicts flow fields, substituting the computationally intensive pixel correspondence matching with a learned model that provides both accuracy and speed.
Solution Approach 2:
The system performs preliminary action by training the neural network on labeled optical flow data beforehand. This pre-training allows the network to learn complex motion patterns in advance, so that during actual optical flow calculation, the system can quickly apply the learned knowledge without performing time-consuming pixel-by-pixel analysis.
2Measurement precision
If traditional pixel correspondence methods are used, then exact pixel intensity correspondence can be sought, but the solution is not unique and determination of optical flow is frustrated or prevented
Solution Approach 1:
The patent substitutes the unreliable pixel intensity matching mechanism with a neural network that learns robust motion patterns. The network processes spatial and temporal information holistically, avoiding the ambiguity of individual pixel correspondence and producing unique, reliable optical flow solutions even in challenging scenarios.
Solution Approach 2:
The neural network acts as an intermediary between input image sequences and optical flow output. It transforms the ambiguous pixel correspondence problem into a well-defined pattern recognition task, mediating the transformation from raw images to reliable flow fields through learned representations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A generative adversarial network (GAN) system 100 includes a generator sub-network 102 configured to examine images 108, 110 of an object 112 moving relative to a viewer of the object 112. The generator sub-network 102 also is configured to generate one or more distribution-based images 300, 302, 304 based on the images 108, 110 that were examined. The system 100 also includes a discriminator sub-network 104 configured to examine the one or more distribution-based images 300, 302, 304 to determine whether the one or more distribution-based images 300, 302, 304 accurately represent the object 112. A predicted optical flow of the object 112 is represented by relative movement of the object 112 as shown in the one or more distribution-based images 300, 302, 304.