Image Sensor DNN Partitioning for Low-Data Private Recognition

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing image sensors output large amounts of data and pose risks to personal information protection.

Innovation Solution

A solid-state image capturing system with a first DNN processing unit on the image sensor and a second DNN processing unit on an information processing device, dividing the DNN algorithm execution to reduce data output and protect personal information.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the image sensor outputs the target image directly, then the identification properties are improved, but the data amount to be output increases and personal information protection is compromised

Engineering Contradiction:

Improveidentification propertiesVSAvoiddata amount

Core Design Contradiction:

Measurement precisionVSQuantity of substance

Solution Approach 1:

The patent divides the DNN processing into two segments: the first DNN on the image sensor performs initial processing to generate intermediate data, and the second DNN on the information processing device completes the processing. This segmentation allows the image sensor to output processed intermediate data instead of the complete target image, reducing data transmission while maintaining identification accuracy.

Inventive Principle:

Principle #1Segmentation

2Measurement precision

If the image sensor outputs the target image directly, then the identification properties are improved, but personal information protection is compromised

Engineering Contradiction:

Improveidentification propertiesVSAvoidpersonal information protection

Core Design Contradiction:

Measurement precisionVSObject-affected harmful factors

Solution Approach 1:

The patent extracts only the necessary intermediate data from the image processing pipeline at the sensor level, rather than outputting the complete target image. The first DNN processes the image data and outputs intermediate results that contain the essential features for identification while removing or obscuring personal information, which is then completed by the second DNN on the information processing device.

Inventive Principle:

Principle #2Taking out (Extraction)

3Productivity

If a DNN function is provided to the image sensor to execute advanced processes, then the processing capability is improved, but the data amount to be output increases

Engineering Contradiction:

Improveprocessing capabilityVSAvoiddata amount

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The first DNN on the image sensor performs preliminary processing of the image data before output, generating intermediate data that has already undergone significant processing. This preliminary action reduces the amount of raw data that needs to be transmitted and processed further, while still maintaining the capability to execute advanced DNN processes at the sensor level.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS12499356B2Solid-state image capturing system, solid-state image capturing device, information processing device, image processing method, and information processing method

Publication Date: 2025.12.16 SONY SEMICON SOLUTIONS CORP

AI summary

Provided is a solid-state image capturing system that includes a solid-state image capturing device and an information processing device. The solid-state image capturing device includes a first DNN processing unit that executes, on image data, a part of a DNN algorithm by a first DNN to generate a first result. The information processing device includes a second DNN processing unit that executes, on the first result acquired from the solid-state image capturing device, remaining of the DNN algorithm by a second DNN to generate a second result.