On-Sensor Image Processing With ML-Based Feature Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image sensors face challenges in efficiently detecting and editing alterable features in digital images before they are viewed by users, leading to potential irritation and inefficient image processing across multiple systems.
Innovation Solution
An intelligent image sensor with an on-sensor controller that utilizes a machine-learning model to detect alterable features and transform or mask them within the sensor before exporting the image data, thereby preventing the transmission of undesirable features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If image data is exported from the sensor without on-sensor processing, then the image processing workload is distributed across multiple external systems, but this leads to inefficient processing and allows alterable features to reach users causing irritation
Solution Approach 1:
The patent combines the image processing functionality with the image sensor by integrating a controller and machine-learning model directly into the sensor device. This merging allows the sensor to perform feature detection and image transformation operations locally, preventing alterable features from being exported without requiring complete redistribution of processing workload across multiple external systems
Solution Approach 2:
The controller performs preliminary processing of the image data by detecting alterable features and transforming the image to mask or remove these features before the image data is exported from the sensor. This preliminary action ensures that undesirable features are eliminated at the source rather than requiring post-processing across multiple external systems
2Object-affected harmful factors
If an on-sensor controller with machine-learning model is integrated to detect and transform alterable features, then user irritation from viewing alterable features is prevented, but the device complexity and processing overhead increase
Solution Approach 1:
The controller acts as an intermediary between the image sensor and external systems, receiving image data from the sensor, processing it through a machine-learning model to detect alterable features, and exporting transformed image data that masks or removes these features. This intermediary approach prevents harmful alterable features from reaching users while maintaining a structured processing architecture
Solution Approach 2:
The image sensor performs self-service by incorporating its own processing capabilities through the controller and machine-learning model. The sensor autonomously detects alterable features in its captured images and transforms the image data to eliminate these features without requiring external intervention, thereby preventing user irritation while managing its own processing needs
3Reliability
If extensive image editing is performed across multiple external systems, then alterable features can be removed, but this increases processing time and computational resources required
Solution Approach 1:
The controller performs the necessary image editing operations preliminarily, detecting alterable features and transforming the image data to mask or remove these features before export. This preliminary action eliminates the need for extensive post-processing across multiple external systems, significantly reducing the time and computational resources required while maintaining effective alterable feature removal
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution effectively prevents users from viewing alterable features, reduces the need for extensive image editing across multiple systems, and enhances the operational efficiency of image sensors and related applications.
Implementation Method 1
Each pixel cell may include a photodiode to sense light by converting photons into charge (e.g., electrons or holes)
Data Source
AI summary
In some examples, a sensor apparatus comprises: an array of pixel cells each including one or more photodiodes configured to generate a charge in response to light, and a charge storage device to convert the charge to output a voltage of an array of voltages, one or more an analog-to-digital converter (ADC) configured the convert the array of voltages to first pixel data, and an on-sensor controller configured to input the first pixel data into a machine-learning model to generate output data comprising prediction data associated with one or more features of the first pixel data, generate, based on the prediction data, second pixel data, the second pixel data associated with one or more transformed features of the first pixel data, and send, from the sensor apparatus to a separate receiving apparatus, the second pixel data.