Neural Network Rectification Layer for Multiscopic Depth Calibration

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Multiscopic image capture systems used by robots face challenges in accurately determining depth maps due to matching errors and lack of runtime feedback, leading to suboptimal calibration and reduced accuracy in depth perception.

Innovation Solution

A neural network-based method that determines a disparity map from multiple images, calculates matching errors, and back-propagates these errors to the rectification layer to update weights, allowing for real-time recalibration of the image capture system during operation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional calibration methods are used for multiscopic image capture systems, then the system can be calibrated initially, but the calibration cannot be updated during runtime leading to reduced accuracy over time

Engineering Contradiction:

Improvedepth map accuracyVSAvoidcalibration validity period

Core Design Contradiction:

Measurement precisionVSDuration of action of stationary object

Solution Approach 1:

The system calculates matching errors from disparity maps and uses these errors as feedback to iteratively update rectification layer weights through back-propagation, enabling continuous calibration improvement during runtime operations

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The calibration system performs self-updating by automatically calculating its own matching errors and using these errors to adjust its rectification parameters without external intervention, maintaining continuous optimization

Inventive Principle:

Principle #25Self-service

2Measurement precision

If matching errors are calculated and back-propagated to update rectification weights, then depth map accuracy improves continuously, but computational complexity increases

Engineering Contradiction:

Improvedisparity map accuracyVSAvoidcalibration computation complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system performs calibration updates selectively based on matching error thresholds rather than continuously, and updates only the rectification layer weights rather than the entire neural network, reducing computational overhead while maintaining accuracy improvements

Inventive Principle:

Principle #16Partial or excessive action

3Adaptability or versatility

If the rectification layer weights are updated during runtime, then the system adapts to changing conditions, but the system requires continuous feedback which increases processing load

Engineering Contradiction:

Improveruntime calibration adaptabilityVSAvoidprocessing energy consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The system performs calibration updates at periodic intervals or based on trigger conditions such as significant matching error accumulation, rather than continuously, reducing processing energy while maintaining adaptability to changing environmental conditions

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS11818328B2Systems and methods for automatically calibrating multiscopic image capture systems

Publication Date: 2023.11.14 GDM HOLDING LLC

AI summary

A method includes receiving, from a multiscopic image capture system, a plurality of images depicting a scene. The method includes determining, by application of a neural network based on the plurality of images, a disparity map of the scene. The neural network includes a plurality of layers, and the layers include a rectification layer. The method include determining a matching error of the disparity map based on differences between corresponding pixels of two or more images associated with the disparity map. The method includes back-propagating the matching error to the rectification layer of the neural network. Back-propagating the matching error includes updating one or more weights applied to the rectification layer.