On-Sensor Model Relearning for Privacy-Safe Image Inference

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The precision of inference results using machine learning models is compromised due to differences between training and operational image data, and relearning on external servers like cloud servers increases communication load and privacy risks.

Innovation Solution

A sensor apparatus with a pixel array unit, storage unit, inference processing unit, and relearning processing unit performs relearning of the machine learning model within the sensor apparatus.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If relearning is performed on external cloud servers, then the machine learning model can be updated, but communication amount increases and privacy risks increase

Engineering Contradiction:

Improvemodel update capabilityVSAvoidcommunication amount

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent extracts the relearning processing from external cloud servers and relocates it to the camera apparatus itself. The storage unit stores the machine learning model locally, and the relearning processing unit performs updates using image data captured by the camera, eliminating the need to transmit large amounts of image data to external servers for relearning.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The camera apparatus performs self-relearning using its own captured image data. The relearning processing unit utilizes image data from the camera's pixel array unit to update the machine learning model stored in the storage unit, enabling the system to maintain and improve its inference capabilities independently without relying on external cloud processing resources.

Inventive Principle:

Principle #25Self-service

2Adaptability or versatility

If relearning is performed on external cloud servers, then the machine learning model can be updated, but privacy risks increase

Engineering Contradiction:

Improvemodel update capabilityVSAvoidprivacy risk

Core Design Contradiction:

Adaptability or versatilityVSObject-affected harmful factors

Solution Approach 1:

The patent removes image data from external cloud servers by performing relearning locally within the camera apparatus. The storage unit stores the machine learning model locally, and the relearning processing unit performs updates using image data captured by the camera, eliminating the need to transmit large amounts of image data to external servers for relearning.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

Inventive Principle:

Principle #25Self-service

3Quantity of substance

If relearning is performed within the sensor apparatus, then communication load is reduced and privacy is protected, but device complexity increases

Engineering Contradiction:

Improvecommunication amountVSAvoidsensor apparatus structure

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent integrates multiple functions into the camera apparatus: the storage unit stores both image data and the machine learning model, while the relearning processing unit performs both relearning processing and inference processing using the same hardware resources. This multi-functionality approach reduces the need for separate dedicated components, thereby limiting the increase in device complexity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentEP4723660A1Sensor device, program, and information processing method

Publication Date: 2026.04.08 SONY SEMICON SOLUTIONS CORP

Figure 1
Figure 2
Figure 3

AI summary

A sensor apparatus according to the present technology includes: a pixel array unit in which pixels each including a light-receiving portion are arrayed two-dimensionally; a storage unit which stores a machine learning model that performs inference processing based on pixel data output from the pixel array unit; an inference processing unit which performs the inference processing using the machine learning model; and a relearning processing unit which performs relearning processing for updating the machine learning model.