Middle Frame Image Processing Multiplexer for Multi-Sensor Systems

Solution Overview

Problem

Conventional imaging apparatuses with multiple image sensors face inefficiencies due to the use of programmable logic devices, which introduce system delay, complexity, cost, and size issues, as well as requiring dynamic switching and re-initialization of image sensors.

Innovation Solution

The implementation of a middle frame image processing apparatus that includes multiple image sensors configured to output middle frame image data objects based on a shared middle frame image size, utilizing an image frame multiplexer to select and process these images without the need for programmable logic devices.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If programmable logic devices (CPLD/FPGA) are used for image processing, then image processing capability is provided, but system delay increases and device complexity increases

Engineering Contradiction:

Improveimage processing capabilityVSAvoidsystem delay

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent extracts the image processing function from the programmable logic device and implements it directly in the image sensor hardware. The image sensor includes dedicated processing circuits that perform image processing operations without requiring external programmable logic devices, thereby eliminating the system delay associated with data transmission and processing in CPLD/FPGA while maintaining image processing capability

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent merges the image processing functions with the image sensor by integrating processing circuits directly into the sensor hardware. This combination allows the image sensor to perform both image capture and processing operations internally, eliminating the need for separate programmable logic devices and reducing overall system complexity and delay

Inventive Principle:

Principle #5Merging (Combining)

2Productivity

If programmable logic devices are used for image processing, then image processing capability is provided, but device complexity and size increase

Engineering Contradiction:

Improveimage processing capabilityVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent extracts the image processing function from the programmable logic device and implements it directly in the image sensor hardware. The image sensor includes dedicated processing circuits that perform image processing operations without requiring external programmable logic devices, thereby eliminating the system delay associated with data transmission and processing in CPLD/FPGA while maintaining image processing capability

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent merges the image processing functions with the image sensor by integrating processing circuits directly into the sensor hardware. This combination allows the image sensor to perform both image capture and processing operations internally, eliminating the need for separate programmable logic devices and reducing overall system complexity and delay

Inventive Principle:

Principle #5Merging (Combining)

3Adaptability or versatility

If dynamic switching between image sensors is implemented, then multi-sensor functionality is achieved, but re-initialization delay occurs

Engineering Contradiction:

Improvemulti-sensor functionalityVSAvoidre-initialization delay

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent implements preliminary action by maintaining all image sensors in a continuously active state with persistent memory buffers that retain image data. When switching between sensors is required, the system can immediately access and process data from any sensor without requiring re-initialization sequences, as all sensors are already powered and configured in advance

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent ensures continuity of useful action by keeping all image sensors operational simultaneously with continuous data acquisition and buffering. This allows seamless switching between sensors without interruption or re-initialization, as the system maintains continuous image capture and processing capabilities across all sensors at all times

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentEP4181518B1Apparatuses and computer-implemented methods for middle frame image processing

Publication Date: 2025.01.29 HAND HELD PRODS INC

Figure 1
Figure 2
Figure 3

AI summary

Embodiments described herein provide a middle frame image processing schema in a multi-sensor environment, for example improve image processing throughput and snappiness by processing middle frame image data objects of a middle frame image size that is shared amongst a plurality of image sensors. An example apparatus includes a plurality of image sensors, each configured to capture a full frame image data object and output a middle frame image data object. The apparatus further includes an image frame multiplexer communicable with each of the plurality of image sensors, and configured to receive a plurality of middle frame image data objects, and output a selected middle frame image data object from the plurality of middle frame image data objects. The apparatus further includes a processor communicable with the image frame multiplexer, and configured to receive the selected middle frame image data object, and process the selected middle frame image data object.