Metadata Generation for Image Recognition Processing Load Reduction

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Application processors face significant information processing burdens when recognizing subjects from input images using neural networks, leading to performance restrictions due to interface limitations and high memory bandwidth requirements.

Innovation Solution

An information processing device that generates metadata to assist the application processor, reducing the processing load by pre-processing image data and selectively outputting relevant information, such as subject positions and attributes, and incorporating watermarking to track image sources.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If an application processor performs subject recognition using a neural network, then recognition accuracy is improved, but information processing amount and memory bandwidth requirements increase significantly

Engineering Contradiction:
Improverecognition accuracyVSAvoidinformation processing amount
Core Design Contradiction:
Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent applies preliminary action by performing object detection and generating metadata about subjects (positions, attributes, classes) before the application processor executes the neural network for recognition. This pre-processing reduces the data volume and complexity that the application processor must handle, thereby maintaining recognition accuracy while significantly reducing information processing amount and memory bandwidth requirements.

Inventive Principle:
Principle #10Preliminary action

2Measurement precision

If an application processor performs comprehensive image processing, then recognition accuracy is improved, but processing speed decreases due to enormous computation requirements

Engineering Contradiction:
Improverecognition accuracyVSAvoidprocessing speed
Core Design Contradiction:
Measurement precisionVSSpeed

Solution Approach 1:

The patent applies segmentation by dividing the image processing workflow into distinct segments: a first processor performs object detection and generates metadata, while the application processor performs neural network recognition based on this metadata. This segmentation allows computationally intensive detection tasks to be handled separately, enabling the application processor to focus on recognition with reduced data volume, thereby improving overall processing speed while maintaining accuracy.

Inventive Principle:
Principle #1Segmentation

3Measurement precision

If all image data is processed for subject recognition, then recognition accuracy is improved, but processing time increases

Engineering Contradiction:
Improverecognition accuracyVSAvoidprocessing time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies the extraction principle by extracting only the essential information from the image data through metadata generation. The metadata contains subject positions, attributes, and detection results that are sufficient for recognition tasks. By extracting and transmitting only this relevant information rather than processing all image data, the system maintains recognition accuracy while significantly reducing processing time.

Inventive Principle:
Principle #2Taking out (Extraction)

Data Source

PatentUS12087049B2Information processing device and information processing method
Publication Date: 2024.09.10 SONY SEMICON SOLUTIONS CORP
  • US12087049B2 patent drawing
  • US12087049B2 patent drawing
  • US12087049B2 patent drawing

AI summary

An information processing device and an information processing method capable of reducing an information processing amount of an application processor that recognizes a subject from an input image are provided. An information processing device according to the present disclosure includes an acquisition unit, a generation unit, and an output unit. From an imaging unit that captures an image and generates image data, the acquisition unit acquires the image data. The generation unit generates, from the image data acquired by the acquisition unit, metadata to assist an application processor that executes processing related to the image. The output unit outputs the metadata generated by the generation unit to the application processor.