Learned Composition Model for Image Cropping

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional image cropping techniques rely on manual parameter adjustments and limited data-driven methods, which fail to account for overall image composition and often remove important features, resulting in poor visual outcomes due to varying user preferences and composition styles.

Innovation Solution

The implementation of combined composition and change-based models for image cropping, utilizing a learned composition model that includes salient regions, foreground detection, and imaging models like Gaussian mixture models and regression functions to evaluate and adjust image composition, maintaining user preferences and improving cropping accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If manual parameter adjustments and hand-crafted cropping rules are used, then cropping can be performed with basic composition guidelines, but the process becomes tedious and cannot accommodate varying user preferences and composition styles

Engineering Contradiction:

Improveaccommodation of varying user preferences and composition stylesVSAvoidtedious parameter adjustments for cropping

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The system performs automatic image cropping without requiring manual parameter adjustments. The learning system autonomously analyzes the image, identifies salient regions, and determines optimal crop boundaries based on learned composition rules, eliminating the need for user intervention in parameter setting while adapting to different composition styles.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent transforms fixed hand-crafted cropping rules into dynamic learned parameters. By training the system on diverse images with various composition styles, the cropping parameters (such as crop boundaries, aspect ratios, and region weights) become adaptive and can vary based on the specific image content and detected composition preferences.

Inventive Principle:

Principle #35Parameter changes

2Manufacturing precision

If data-driven methods only model composition changes before and after cropping, then some composition improvements can be achieved, but important image features are removed and overall composition is not properly considered

Engineering Contradiction:

Improvecropping accuracyVSAvoidremoval of important image features

Core Design Contradiction:

Manufacturing precisionVSLoss of information

Solution Approach 1:

The system performs preliminary analysis of the image to identify salient regions and important features before executing the crop. By detecting key objects, regions of interest, and composition elements in advance, the system ensures that these important features are preserved in the cropped output rather than being removed by blind application of composition rules.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The learning system incorporates feedback loops where the detected salient regions and composition quality metrics are used to adjust and refine the cropping decision. The system evaluates how well different crop options preserve important features and adjusts the crop boundaries to maximize both composition quality and feature retention.

Inventive Principle:

Principle #23Feedback

3Adaptability or versatility

If conventional cropping techniques are used, then simple composition adjustments can be made, but the overall composition quality and visual outcome are poor due to inability to account for varying user preferences

Engineering Contradiction:

Improveuser-specific cropping stylesVSAvoidcropping accuracy

Core Design Contradiction:

Adaptability or versatilityVSManufacturing precision

Solution Approach 1:

The patent implements dynamic cropping that adapts to different user preferences and image types. Rather than applying static composition rules, the system dynamically adjusts cropping parameters based on the specific image content, detected objects, and learned user preferences, allowing the same system to produce appropriate crops for portraits, landscapes, and other genres with different composition requirements.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS10019823B2Combined composition and change-based models for image cropping

Publication Date: 2018.07.10 ADOBE INC

AI summary

In techniques of combined composition and change-based models for image cropping, a composition application is implemented to apply one or more image composition modules of a learned composition model to evaluate multiple composition regions of an image. The learned composition model can determine one or more cropped images from the image based on the applied image composition modules, and evaluate a composition of the cropped images and a validity of change from the image to the cropped images. The image composition modules of the learned composition model include a salient regions module that iteratively determines salient image regions of the image, and include a foreground detection module that determines foreground regions of the image. The image composition modules also include one or more imaging models that reduce a number of the composition regions of the image to facilitate determining the cropped images from the image.