UHD Sensor Data Processing Apparatus for Real-Time Frame Alignment

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional video architectures struggle to support the bandwidth and timing requirements of ultra-high definition (UHD) image sensors, leading to inefficient data transport and loss of raw data, as they are often based on legacy HD video standards that are not capable of handling the higher pixel density and data volume of UHD sensors.

Innovation Solution

A UHD sensor data processing apparatus and method that divides raw UHD data into lossless segments, processes them in parallel using existing HD video architectures, and uses metadata to reconstruct the original data, enabling efficient transport of UHD video data by leveraging pixel packing methods and multiple physical connections, while also allowing for real-time stitching of panoramic imagery using geolocation and inertial data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If legacy HD video architectures are used to transport UHD video data, then existing infrastructure can be utilized, but bandwidth and timing requirements of UHD sensors cannot be met

Engineering Contradiction:

Improvecompatibility with existing infrastructureVSAvoidbandwidth and timing capability

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent divides UHD video frames into multiple segments that can be transported through existing HD video infrastructure. Each segment is processed independently and then reassembled, allowing the system to leverage legacy hardware while supporting UHD resolution requirements.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces temporal dimension by processing multiple UHD frames simultaneously across parallel time periods. This allows the system to multiplex UHD data through HD infrastructure by utilizing time as an additional dimension for data transport.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Quantity of substance

If video compression techniques are used to transport UHD data, then bandwidth requirements are reduced, but raw sensor data is lost or discarded

Engineering Contradiction:

Improvedata volume reductionVSAvoidraw data integrity

Core Design Contradiction:

Quantity of substanceVSLoss of information

Solution Approach 1:

The patent extracts only the essential metadata and timing information needed for synchronization, while preserving the complete raw sensor data. This allows the system to reduce processing overhead without discarding any original information.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent creates copies of UHD video data organized into time-period segments that can be independently transported through existing infrastructure. These segmented copies maintain data integrity while enabling efficient transport through parallel processing paths.

Inventive Principle:

Principle #26Copying

3Reliability

If new video architectures are developed from scratch for UHD, then bandwidth and timing requirements are met, but existing HD video infrastructure cannot be utilized

Engineering Contradiction:

Improvebandwidth and timing capabilityVSAvoidsystem development cost

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent creates a universal processing architecture that can handle both UHD and HD video data using the same infrastructure. The system processes multiple UHD frames simultaneously while maintaining compatibility with existing HD video equipment and standards.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent implements dynamic frame segmentation and reassembly processes that adapt to different video formats and resolutions. This dynamic approach allows the same hardware infrastructure to efficiently handle both legacy HD and modern UHD data without requiring separate dedicated systems.

Inventive Principle:

Principle #15Dynamics

4Adaptability or versatility

If UHD video data is split into multiple packets or sub-frames for transport, then existing HD architectures can be used, but real-time display is prevented due to additional processing steps

Engineering Contradiction:

Improvecompatibility with HD architectureVSAvoidprocessing latency

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent performs preliminary segmentation and metadata extraction operations during the capture phase, preparing data for efficient transport. By pre-processing and organizing data into time-period segments before transport, the system reduces processing delays during transmission and reconstruction.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10412395B2Real time frame alignment in video data

Publication Date: 2019.09.10 RAYTHEON CO

AI summary

A sensor data processing apparatus can be coupled to multiple image sensors of different types. The apparatus determines a geolocation of areas imaged in each frame or pixel based on sensor geolocation information and encodes the geolocation information in metadata space of the corresponding frame. The apparatus is configured to align edges and/or corners of the frame with edges or corners of adjacent frames based on the geolocation information to stitch together a mosaic or panoramic image without overlapping frames. The apparatus can be figured to stitch together image frame in real time without performing feature registration and without resampling frames.