Solid-State Imaging With Split DNN Processing for Privacy

Solution Overview

Problem

Existing image sensors output large data amounts and lack effective protection for personal information, necessitating a solution that reduces data output while ensuring privacy.

Innovation Solution

A solid-state image capturing system with a dual-DNN processing architecture, where a first DNN processing unit on the image sensor performs initial object recognition, followed by a second DNN processing unit on an information processing device, reducing data transmission and protecting personal information through encryption.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If an image sensor outputs target images for advanced processing, then image processing capability is improved, but data amount to be output increases and personal information protection is compromised

Engineering Contradiction:

Improveimage processing capabilityVSAvoiddata amount

Core Design Contradiction:

ProductivityVSQuantity of substance

Solution Approach 1:

The patent extracts only the necessary feature information from the image data through DNN processing, outputting only the extracted features rather than the complete image data. This extraction principle reduces the data amount while preserving the essential information needed for advanced processing, thereby resolving the contradiction between processing capability and data volume.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent segments the DNN processing into two parts: initial processing performed by the image sensor to extract features, and subsequent processing performed by an external device. This segmentation allows the image sensor to handle only the feature extraction portion, reducing its output data amount while still enabling advanced processing capabilities through the combined system.

Inventive Principle:

Principle #1Segmentation

2Productivity

If an image sensor outputs target images for advanced processing, then image processing capability is improved, but personal information protection is compromised

Engineering Contradiction:

Improveimage processing capabilityVSAvoidpersonal information exposure

Core Design Contradiction:

ProductivityVSObject-affected harmful factors

Solution Approach 1:

The patent extracts only the necessary feature information from the image data through DNN processing, outputting only the extracted features rather than the complete image data. This extraction principle reduces the data amount while preserving the essential information needed for advanced processing, thereby resolving the contradiction between processing capability and data volume.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent segments the DNN processing into two parts: initial processing performed by the image sensor to extract features, and subsequent processing performed by an external device. This segmentation allows the image sensor to handle only the feature extraction portion, reducing its output data amount while still enabling advanced processing capabilities through the combined system.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentEP3855726B9Solid-state imaging system, solid-state imaging device, information processing device, image processing method, information processing method, and program

Publication Date: 2025.09.17 SONY SEMICON SOLUTIONS CORP

Figure 1
Figure 2
Figure 3

AI summary

A solid-state image capturing system (1) includes a solid-state image capturing device (100) and an information processing device (200). The solid-state image capturing device (100) includes a first DNN processing unit (130) that executes, on image data, a part of a DNN algorithm by a first DNN to generate a first result. The information processing device (200) includes a second DNN processing unit (230) that executes, on the first result acquired from the solid-state image capturing device, remaining of the DNN algorithm by a second DNN to generate a second result.