Machine-Learned Video Frame Selection Through Aesthetic Crop Scoring

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional image cropping methods are tedious, burdensome, and inefficient, especially when dealing with large volumes of images, and they lack the ability to apply artistic and aesthetic judgment automatically.

Innovation Solution

The use of machine learning (ML) predictors to automate the image cropping process by training the ML predictor program with a plurality of training raw images and their associated master images, allowing it to predict cropping characteristics and determine the highest statistical confidence scores for video frame selection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional image cropping methods are used, then manual control over cropping decisions is maintained, but the process becomes tedious and inefficient when dealing with large volumes of images

Engineering Contradiction:
Improveimage processing throughputVSAvoidmanual cropping effort
Core Design Contradiction:
ProductivityVSEase of operation

Solution Approach 1:

The system employs machine learning models that automatically analyze images and determine optimal cropping regions without human intervention. The model independently evaluates multiple candidate crops, scores them based on learned aesthetic and compositional criteria, and selects the best crop automatically, enabling the system to serve itself rather than requiring manual operation for each image.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent replaces manual mechanical cropping operations with an automated machine learning-based system. Instead of human operators visually inspecting and cropping images manually, the system uses trained neural networks to perform the same function automatically, substituting the mechanical human-in-the-loop process with an automated computational approach that significantly increases throughput.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If conventional automated cropping methods are used, then processing speed increases, but the ability to apply artistic and aesthetic judgment is lost

Engineering Contradiction:
Improveautomated processing speedVSAvoidaesthetic quality of crop selection
Core Design Contradiction:
ProductivityVSManufacturing precision

Solution Approach 1:

The system performs preliminary training of machine learning models using large datasets of images with known good crops. During this training phase, the model learns aesthetic principles, compositional rules, and artistic judgment criteria from examples. This preliminary learning enables the model to apply acquired knowledge to new images, maintaining aesthetic quality while achieving automated processing speed.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent transforms the abstract concept of aesthetic judgment into quantifiable parameters that machine learning models can process. By converting qualitative artistic criteria into numerical scores and mathematical representations, the system enables automated systems to evaluate and compare multiple crop options based on learned aesthetic parameters, thereby maintaining quality while achieving automation.

Inventive Principle:
Principle #35Parameter changes

3Manufacturing precision

If multiple cropping characteristics are evaluated for each video frame, then the quality of frame selection improves, but the computational complexity increases

Engineering Contradiction:
Improveframe selection accuracyVSAvoidcomputational processing requirements
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The system segments the frame selection process into distinct computational stages: extracting visual features from frames, generating multiple candidate crops per frame, scoring each candidate using trained models, and selecting the best frame based on aggregated scores. This segmentation allows complex multi-criteria evaluation to be broken down into manageable computational steps, improving accuracy while controlling complexity.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS20250292531A1Selection of Video Frames Using a Machine Learning Predictor
Publication Date: 2025.09.18 GRACENOTE INC
  • US20250292531A1 patent drawing
  • US20250292531A1 patent drawing
  • US20250292531A1 patent drawing

AI summary

Example systems and methods of selection of video frames using a machine learning (ML) predictor program are disclosed. The ML predictor program may generate predicted cropping boundaries for any given input image. Training raw images associated with respective sets of training master images indicative of cropping characteristics for the training raw image may be input to the ML predictor, and the ML predictor program trained to predict cropping boundaries for raw image based on expected cropping boundaries associated training master images. At runtime, the trained ML predictor program may be applied to a sequence of video image frames to determine for each respective video image frame a respective score corresponding to a highest statistical confidence associated with one or more subsets of cropping boundaries predicted for the respective video image frame. Information indicative of the respective video image frame having the highest score may be stored or recorded.