Image Skew Rectification and Cleanup for Productivity

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Captured photographic images from mobile devices often suffer from perspective skews and unwanted regions due to physical limitations and less-than-ideal lighting conditions, requiring rectification and cleanup before they can be effectively used in productivity applications.

Innovation Solution

A computing device with image processing capabilities that detects image edges, corrects skew states, applies image cleanup algorithms, and classifies images to adjust and enhance their quality, specifically using edge detection, cleanup, and classification modules to rectify and improve the readability of whiteboard, document, and blackboard images.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If photographs are taken at an angle due to physical dimension limitations, then the photograph can be captured in constrained spaces, but perspective skew is introduced that requires rectification

Engineering Contradiction:
Improveability to capture photograph in constrained spaceVSAvoidimage geometric accuracy
Core Design Contradiction:
Ease of operationVSManufacturing precision

Solution Approach 1:

The system performs preliminary edge detection and skew detection on the captured image, then applies geometric transformation to rectify the perspective distortion before the image is used for document scanning or processing. This preliminary correction eliminates the need for manual adjustment and enables accurate processing despite the angled capture position.

Inventive Principle:
Principle #10Preliminary action

2Ease of operation

If photographs are taken in less than ideal lighting conditions, then the photograph can be captured in meeting room environments, but reflections and glare are introduced that require cleanup

Engineering Contradiction:
Improveability to capture photograph in meeting room environmentVSAvoidimage quality
Core Design Contradiction:
Ease of operationVSManufacturing precision

Solution Approach 1:

The system performs preliminary analysis of the captured image to detect regions with reflections, glare, or poor lighting conditions. It then applies targeted image processing algorithms to correct these issues before the image is processed further, thereby eliminating the need for manual cleanup and enabling reliable automated processing.

Inventive Principle:
Principle #10Preliminary action

3Manufacturing precision

If image processing algorithms are applied to correct skew and cleanup reflections, then image quality is improved, but processing time is increased

Engineering Contradiction:
Improveimage qualityVSAvoidimage processing time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The image processing system divides the captured image into multiple regions and applies different processing algorithms to each region based on its characteristics. For example, edge detection is applied to boundary regions, while reflection removal is applied only to regions containing glare. This segmented approach reduces overall processing time while maintaining image quality.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system applies image processing algorithms selectively rather than uniformly across the entire image. It identifies and processes only the regions that require correction (e.g., skewed edges, reflective areas), leaving already-quality regions unchanged. This partial action approach significantly reduces processing time while maintaining high image quality where needed.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentEP3069502B1Image processing for productivity applications
Publication Date: 2022.08.24 MICROSOFT TECHNOLOGY LICENSING LLC
  • EP3069502B1 patent drawingFigure 1A
  • EP3069502B1 patent drawingFigure 1B
  • EP3069502B1 patent drawingFigure 2

AI summary

Image processing for productivity applications is provided. An image may be received by a computing device. The computing device may detect the edges comprising the received image and adjust the image based on a skew state of the detected edges. The computing device may then process the adjusted image to correct imbalances. The computing device may then assign an image classification to the processed image. The computing device may then adjust the processed image based on the assigned image classification.