Synthetic Optical Flow Data Generation via Proxy Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Collecting sufficient optical flow labels in a real-world environment is challenging due to high human resource and monetary costs, necessitating the development of a method for efficiently synthesizing large-scale datasets for optical flow research.
Innovation Solution
A method involving an image change data generating apparatus that inputs an initial image and data parameters to a data generator, generates consecutive frames, and uses pre-trained proxy networks to generate image change data, with loss functions calculated and data parameters updated to minimize and maximize specific loss functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-world optical flow label data is collected, then data quality and reliability are improved, but human resource costs and monetary costs increase significantly
Solution Approach 1:
The patent uses synthetic image data generated by rendering engines as copies of real-world scenarios. These synthetic images with ground truth optical flow labels serve as substitutes for expensive real-world annotated data, maintaining training effectiveness while eliminating the need for costly manual annotation processes
Solution Approach 2:
The patent extracts only the essential elements needed for optical flow training (image pairs with known transformations) and synthesizes them computationally. By taking out the expensive annotation step and replacing it with automated synthetic generation, the method preserves data quality while removing resource bottlenecks
2Productivity
If large-scale optical flow datasets are synthesized, then data availability and productivity are improved, but the effectiveness and quality of training data remain uncertain
Solution Approach 1:
The patent performs preliminary actions by pre-defining transformation parameters (translations, rotations, scaling) and applying them systematically during synthetic image generation. This ensures that ground truth optical flow labels are accurately known beforehand, guaranteeing training data effectiveness while enabling large-scale generation
Solution Approach 2:
The patent systematically varies transformation parameters (displacement vectors, rotation angles, scaling factors) to generate diverse optical flow patterns. By changing these parameters across multiple synthetic scenarios, the method produces varied training data that maintains effectiveness while achieving large scale
3Adaptability or versatility
If multiple proxy networks with different time intervals are used, then adaptability and training data versatility are improved, but device complexity and computational requirements increase
Solution Approach 1:
The patent designs proxy networks that can handle multiple time intervals and transformation types through a unified architecture. The same network structure processes different temporal scenarios by adjusting input parameters, achieving versatility without proportionally increasing complexity
Solution Approach 2:
The patent segments the training process into multiple proxy networks, each specialized for specific time interval ranges. This segmentation allows each network to be relatively simple while the collective system covers diverse temporal scenarios, balancing complexity and adaptability
Data Source
AI summary
There is provided a method for generating image change data to be performed by an image change data generating apparatus, the method comprising, inputting an initial image and data parameters to a data generator, generating a first image and a second image respectively, within a first time interval by using the data generator based on the initial image and the data parameters, the first image and the second image being in a relationship of consecutive frames with each other, and generating a first image change data by using a pre-trained first proxy network and a second image change data by using a pre-trained second proxy network based on the first image and the second image.