Multispectral Image Recognition Device Avoiding Synthesis Artifacts

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional image recognition systems using multispectral images often suffer from reduced subject recognition accuracy due to artifacts present in synthesized data across different wavelength bands.

Innovation Solution

An image recognition device and method that captures images in multiple wavelength bands and uses a deep neural network (DNN) to recognize subjects from each individual piece of image data without synthesizing, thereby avoiding artifacts and improving accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If multispectral images are synthesized by combining data from multiple wavelength bands, then comprehensive spectral information is obtained, but subject recognition accuracy deteriorates due to artifacts in synthesized data

Engineering Contradiction:
Improvespectral information completenessVSAvoidsubject recognition accuracy
Core Design Contradiction:
Loss of informationVSMeasurement precision

Solution Approach 1:

The patent segments the image processing task by performing subject recognition separately for each wavelength band instead of synthesizing all bands first. The recognition unit executes independent recognition processes for each band, avoiding the artifact contamination that occurs during synthesis while still utilizing information from multiple bands through sequential processing.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent extracts and removes the harmful synthesis process from the workflow. By taking out the synthesis step that causes artifacts, the system directly processes individual wavelength band images through the recognition unit, eliminating the source of recognition errors while preserving the ability to process multispectral data.

Inventive Principle:
Principle #2Taking out (Extraction)

2Adaptability or versatility

If conventional synthesis methods are used to create multispectral images, then data integration across wavelength bands is achieved, but artifacts are introduced that reduce recognition accuracy

Engineering Contradiction:
Improvemultispectral data integrationVSAvoidrecognition accuracy
Core Design Contradiction:
Adaptability or versatilityVSReliability

Solution Approach 1:

The patent inverts the conventional workflow by reversing the order of operations. Instead of synthesizing first then recognizing (conventional approach), the system recognizes subjects from individual bands first, then integrates results. This inversion eliminates artifacts while maintaining multispectral capabilities through alternative integration of recognition outcomes.

Inventive Principle:
Principle #13The other way round (Inversion)

3Use of energy by moving object

If synthesized multispectral data is used for recognition, then comprehensive spectral coverage is achieved, but harmful artifacts are introduced that deteriorate recognition performance

Engineering Contradiction:
Improvespectral coverageVSAvoidartifacts in synthesized data
Core Design Contradiction:
Use of energy by moving objectVSObject-affected harmful factors

Solution Approach 1:

The patent extracts and eliminates the harmful synthesis step from the processing chain. By removing the synthesis operation that generates artifacts, the system processes each wavelength band independently through the recognition unit, maintaining comprehensive spectral coverage through multi-band processing while eliminating artifact contamination.

Inventive Principle:
Principle #2Taking out (Extraction)

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

The approach enhances subject recognition accuracy by processing each wavelength band independently with a DNN, reducing the influence of artifacts and improving the overall recognition performance.

Implementation Method 1

imaging pixels receiving light in four or more types of wavelength bands

Methodology Applied
Scientific EffectLight reception by imaging pixels: Photoelectric Effect

Data Source

PatentUS12165402B2Image recognition device and image recognition method
Publication Date: 2024.12.10 SONY SEMICON SOLUTIONS CORP
  • US12165402B2 patent drawing
  • US12165402B2 patent drawing
  • US12165402B2 patent drawing

AI summary

Provided are an image recognition device and an image recognition method capable of improving subject recognition accuracy. The image recognition device (image sensor 1) according to the present disclosure includes an imaging unit (10) and a recognition unit (14). The imaging unit (10) generates image data by capturing a plurality of images in different wavelength bands using imaging pixels (R, G, B, IR) receiving light in four or more types of wavelength bands. The recognition unit (14) recognizes a subject from each of the plurality of pieces of image data for each of the wavelength bands.