Email Image Classification via Facial Recognition Metadata

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current email systems lack the ability to classify attachments based on content, making it difficult and inefficient for users to access and search for specific image files within email messages.

Innovation Solution

An email server is equipped with an image classifier that automatically detects and classifies image files by comparing them with known images, using facial recognition and metadata tagging to identify and tag content, allowing for efficient searching of images within email attachments.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If manual browsing through email messages is used to locate image files, then users can find specific images, but the process is cumbersome and time-consuming

Engineering Contradiction:
Improvetime to locate image filesVSAvoidease of searching for images
Core Design Contradiction:
Loss of timeVSEase of operation

Solution Approach 1:

The system performs preliminary classification of image attachments by automatically detecting, analyzing, and tagging images with metadata (such as content descriptions, facial recognition results, and contextual information) at the time of email receipt. This preliminary action eliminates the need for manual browsing later, as users can directly search for images based on their classified content.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary classification system that acts as a mediator between raw image attachments and user search queries. This system includes components for detecting image files, retrieving known images for comparison, comparing images to identify content, and modifying metadata - effectively bridging the gap between unstructured attachments and structured search capabilities.

Inventive Principle:
Principle #24Intermediary (Mediator)

2Adaptability or versatility

If typical search functions are used that only search message bodies and sender/recipient names, then search operations are simple, but content in attached files cannot be searched

Engineering Contradiction:
Improvesearch functionality coverageVSAvoidsystem complexity
Core Design Contradiction:
Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The classification system is designed to handle multiple types of attachments beyond just images, including documents and other file types. The scanner detects various attachment types, and the system can classify and index content across different formats, making the search functionality universal and applicable to diverse attachment types while maintaining a unified approach.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent introduces an intermediary classification system that acts as a mediator between raw image attachments and user search queries. This system includes components for detecting image files, retrieving known images for comparison, comparing images to identify content, and modifying metadata - effectively bridging the gap between unstructured attachments and structured search capabilities.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Productivity

If automatic image classification is implemented, then user access to image files is improved, but the email server requires additional processing capabilities

Engineering Contradiction:
Improveuser access efficiency to attachmentsVSAvoidemail server complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The classification system is divided into distinct functional modules: a scanner for detecting attachments, a retriever for obtaining known images, a comparator for analyzing image content, and a modifier for updating metadata. This segmentation allows the email server to progressively process images through specialized components, improving productivity while managing complexity through modular design.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS11468707B2Automatic image classification in electronic communications
Publication Date: 2022.10.11 MICROSOFT TECHNOLOGY LICENSING LLC
  • US11468707B2 patent drawing
  • US11468707B2 patent drawing
  • US11468707B2 patent drawing

AI summary

Techniques of automatic image classification and modification in computing systems are disclosed herein. In one embodiment, a method includes scanning an inbox on email servers for emails containing image files. Upon detecting that an email in the inbox contains an image file, the method includes retrieving an identification photo of a user from a data store. The method also includes determining, via facial recognition, whether the image file in the email contains at least a partial image of the user based on the retrieved identification photo. In response to determining that the image file in the email contains at least a partial image of the user, a metadata value is inserted into the image file indicating that the image file contains at least a partial image of the user before the image file is stored in the inbox on the one or more email servers.