Vehicle Camera Depth Readout for Low-Latency Close Object Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing camera systems face challenges in quickly determining depth information from multiple images, particularly for objects close to a vehicle, which is crucial for timely collision avoidance in automated vehicle control systems.

Innovation Solution

The method involves capturing and processing images from a set of cameras by reading image data from the bottom of the scene area first, allowing for the determination of distances to objects in the lower portion of the scene before processing the upper portion, using both global and rolling shutter sensors to reduce reaction time and improve collision avoidance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If image data is processed from top to bottom of the scene area, then complete depth information is available, but depth information for closer objects (lower portion) is obtained later

Engineering Contradiction:

Improvetime to detect close objectsVSAvoidcompleteness of depth information

Core Design Contradiction:

Loss of timeVSReliability

Solution Approach 1:

The patent divides the image processing into segments by prioritizing the lower portion of the scene area first, then processing the upper portion. This segmentation allows depth information for closer objects to be extracted and used earlier, reducing the time loss for critical collision avoidance while maintaining overall processing completeness.

Inventive Principle:

Principle #1Segmentation

2Speed

If depth processing is performed on the entire scene area before any vehicle control operation, then complete depth information is available, but reaction time for collision avoidance is increased

Engineering Contradiction:

Improvereaction speed for collision avoidanceVSAvoidaccuracy of depth information

Core Design Contradiction:

SpeedVSMeasurement precision

Solution Approach 1:

The patent performs preliminary depth processing on the lower portion of the scene area before completing processing of the entire scene. This preliminary action provides sufficient depth information for immediate vehicle control operations and collision avoidance, achieving fast reaction speed while maintaining adequate measurement precision for safety-critical decisions.

Inventive Principle:

Principle #10Preliminary action

3Loss of time

If image data is read out sequentially from the camera sensor, then data is captured in order, but time is lost before depth information for closer objects is available

Engineering Contradiction:

Improvelatency in depth information availabilityVSAvoidthroughput of image processing

Core Design Contradiction:

Loss of timeVSProductivity

Solution Approach 1:

The patent inverts the conventional processing order by reading out and processing image data from the bottom of the scene area first (lower portion corresponding to closer objects), then proceeding to the upper portion. This inversion reduces latency for critical close object detection while maintaining overall processing throughput through efficient parallel processing paths.

Inventive Principle:

Principle #13The other way round (Inversion)

Data Source

PatentUS11827216B2Optimal sensor reading for low latency object detection

Publication Date: 2023.11.28 DEERE & CO

AI summary

Methods and apparatus for matching portions of images are described as well as using depth information generated from the matching results. In various embodiments, lower portions of images of a scene area, which are more likely to include objects closer to a vehicle on which cameras are mounted, than are portions of images corresponding to an upper portion of the scene area, are processed and used for depth determination. In this way, depths to objects, which are likely to be closer to the vehicle than objects in the upper portion of the scene area, are determined first and used to control a vehicle reducing the risk of collision as compared to systems where the depth determination of an entire scene is completed before depth information is used for control purposes with the order of processing being such that nearer objects are likely to be detected prior to more distant objects.