Shared Encoder Neural Network for Scene Flow Estimation

The patent combines separate neural networks for optical flow and disparity estimation into a unified scene flow estimation network. The shared backbone architecture processes both tasks simultaneously, reducing overall system complexity while maintaining estimation accuracy through integrated feature extraction and task-specific head modules.

The unified neural network is designed to perform multiple functions: it estimates both optical flow and disparity through shared feature extraction layers followed by task-specific prediction heads. This multi-functional design reduces memory requirements and computational overhead compared to maintaining separate specialized networks.

The training approach uses a composite dataset combining synthetic data with rich ground-truth annotations and real photorealistic images. This composite training strategy leverages the advantages of both data types: synthetic data provides precise supervision signals while real images ensure photorealism and diverse scene representations, creating a more robust trained model.

The system performs preliminary training or pre-training on synthetic data with rich annotations to establish baseline performance, then fine-tunes on photorealistic real-world data. This staged approach allows the model to first learn from perfectly annotated synthetic scenarios before adapting to the complexity and variability of real-world scenes.

The system enables continuous training by efficiently processing and utilizing all available training data through the unified neural network architecture. The shared feature extraction and multi-task learning framework allow the model to continuously learn from diverse data samples without requiring excessive computational resources, maintaining both training speed and quality.

The unified scene flow estimation network performs self-supervised learning where the model generates its own training signals through consistency constraints and photometric losses. This self-service capability allows the system to effectively utilize limited labeled data while automatically creating additional supervision from unlabeled photorealistic images, improving training quality without proportionally increasing data requirements.