Vehicle Camera Fusion With Event-Triggered Object Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current autonomous vehicle systems require significant processing power to continuously analyze image data from conventional cameras, leading to inefficiencies in processing and communication resources.

Innovation Solution

Integration of an event camera and a conventional camera with a control unit that receives movement data from the event camera, instructs the conventional camera to capture images only when necessary, and analyzes both data types to determine objects of interest, using machine learning and AI models to optimize processing and communication.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional cameras continuously capture images for autonomous vehicle navigation, then safety and operational awareness are improved, but processing power requirements increase significantly

Engineering Contradiction:

Improvevehicle safetyVSAvoidprocessing power

Core Design Contradiction:

ReliabilityVSPower

Solution Approach 1:

The system segments the monitoring task between two camera types: event cameras detect motion events and trigger conventional cameras only when needed. This segmentation allows continuous surveillance with minimal processing by dividing the workload - event cameras handle broad area motion detection while conventional cameras provide detailed imagery only for relevant events.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of continuous image capture and processing, the system uses periodic action by triggering conventional camera capture only at specific moments when motion is detected by event cameras. This transforms continuous processing into event-driven periodic processing, significantly reducing overall processing power requirements while maintaining safety.

Inventive Principle:

Principle #19Periodic action

2Speed

If conventional cameras continuously capture and transmit image data, then real-time analysis capability is improved, but communication resource consumption increases

Engineering Contradiction:

Improvereal-time analysis speedVSAvoidcommunication resource consumption

Core Design Contradiction:

SpeedVSLoss of energy

Solution Approach 1:

The system extracts only the essential information needed for analysis by using event cameras to detect and report only motion events. Instead of transmitting continuous video streams, only motion-triggered image data is captured and transmitted, extracting the minimum necessary data for real-time analysis while minimizing communication overhead.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system applies partial action by capturing images only partially - specifically only when motion events occur - rather than continuously. This selective partial capture provides sufficient data for real-time safety analysis without the excessive communication resources required for continuous full-stream transmission.

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If event cameras are used to detect motion, then processing efficiency is improved, but measurement precision for object identification may be reduced compared to continuous conventional camera capture

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidobject identification accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The system merges the strengths of both camera types: event cameras provide high temporal resolution motion detection with high processing efficiency, while conventional cameras provide high spatial resolution for accurate object identification. By combining these complementary technologies and coordinating their operation, the system achieves both processing efficiency and measurement precision.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

Event cameras perform preliminary action by pre-detecting motion events and triggering conventional cameras only when relevant objects or events are detected. This preliminary motion detection filters out unnecessary captures, ensuring that conventional cameras focus their high-resolution imaging only on potentially relevant targets, thus maintaining identification accuracy while improving overall processing efficiency.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20240326865A1Systems and methods for improving vehicle operation based on data captured from event and conventional cameras

Publication Date: 2024.10.03 HONDA MOTOR CO LTD

AI summary

Provided herein is a vehicle including an event camera coupled to the vehicle, a conventional camera coupled to the vehicle, and a control unit communicatively coupled to the event camera and to the conventional camera. The control unit is configured to: receive, from the event camera when the event camera senses movement of at least one object from the surroundings of the vehicle, movement data associated with the at least one object, instruct, based upon the movement data, the conventional camera to capture an image of the at least one object, and analyze the movement data and the captured image to determine whether the at least one object is an object of interest.