Machine Learning Model for Adaptive Image Region Selection

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image cropping technologies cannot adaptively determine the principal portion of an image based on its intended purpose, as the position of the principal portion varies significantly depending on how the image is used, such as in landscape, portrait, or fashion contexts.

Innovation Solution

An information-processing device and method that utilizes a machine learning model to output regions and evaluation values for an input image, where the output changes based on the attribute of how the image is used, by combining intermediate heatmaps from multiple learned models and using a teacher-student learning approach to efficiently identify the principal portion.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If a single machine learning model is used for image cropping, then the device complexity is reduced, but the adaptability to different image usage purposes deteriorates

Engineering Contradiction:
Improvemodel structureVSAvoidadaptability to different usage purposes
Core Design Contradiction:
Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent segments the single machine learning model into multiple specialized models, each trained to evaluate images for specific usage purposes (e.g., landscape, portrait, fashion). Each model outputs an attention map focused on its specialized domain, and these attention maps are combined to produce the final cropping result. This segmentation allows the system to maintain low device complexity while achieving high adaptability across different image usage contexts.

Inventive Principle:
Principle #1Segmentation

2Adaptability or versatility

If multiple specialized machine learning models are trained for different image purposes, then the adaptability improves, but the learning data preparation workload increases

Engineering Contradiction:
Improveadaptability to different usage purposesVSAvoidlearning data preparation time
Core Design Contradiction:
Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent creates a universal machine learning framework that can be applied across multiple image usage purposes. By training specialized models within this universal framework, the system achieves multi-functionality where the same overall architecture and training methodology can be reused for different purposes (landscape, portrait, fashion, etc.). This universality reduces the learning data preparation workload by allowing the system to leverage the same training pipeline and architectural patterns across different specialized models, rather than creating entirely separate systems for each purpose.

Inventive Principle:
Principle #6Universality (Multi-functionality)

3Measurement precision

If attention maps from multiple models are combined, then the measurement precision of the principal portion improves, but the device complexity increases

Engineering Contradiction:
Improveprecision of principal portion identificationVSAvoidmodel combination structure
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent merges the attention maps produced by multiple specialized machine learning models into a single composite attention map. This merging process combines the strengths of each specialized model's attention focus, resulting in a more precise identification of the principal portion that accounts for the specific image usage purpose. The merging operation integrates the different attention perspectives (landscape, portrait, fashion) to produce a comprehensive and accurate cropping region, improving measurement precision while maintaining manageable device complexity through efficient combination algorithms.

Inventive Principle:
Principle #5Merging (Combining)

Data Source

PatentUS20240362885A1Image scoring apparatus, image scoring method and method of machine learning
Publication Date: 2024.10.31 RAKUTEN GROUP INC
  • US20240362885A1 patent drawing
  • US20240362885A1 patent drawing
  • US20240362885A1 patent drawing

AI summary

Provided is an information-processing device including a CPU; and a memory storing instructions for causing the information-processing device, when executed by the CPU, to: include at least a machine learning model configured to receive an input image and an attribute as input, and to output at least one region in the input image and an evaluation value for each of the at least one region, wherein, for a common input image, the region and the evaluation value output when one attribute is given are different from the region and the evaluation value output when another attribute different from the one attribute is given.