Image Aesthetic Assessment with FCNs and Region Composition Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated image aesthetics assessment systems are sensitive to feature detector inaccuracies and fail to consider image composition, leading to suboptimal results.
Innovation Solution
Utilizing fully convolutional networks (FCNs) and region composition graphs to generate 3D feature maps, model mutual dependencies between image features, and perform graph convolution to determine aesthetic appeal, incorporating atrous spatial pyramid pooling and specific feature encoders like DenseNet to preserve fine-grained details.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional feature detectors are used to detect objects in an image, then the system can identify image regions, but inaccuracies in detection destroy the composition information of the entire region collection
Solution Approach 1:
The patent introduces Fully Convolutional Networks (FCNs) as an intermediary between traditional feature detectors and composition analysis. FCNs generate accurate 3D feature maps that preserve spatial relationships and composition information, mediating the transition from raw image data to reliable region composition representation without the inaccuracies of traditional detectors
Solution Approach 2:
The patent replaces traditional mechanical feature detection methods with neural network-based FCNs. This substitution enables the system to automatically learn and extract accurate image features while preserving composition information, overcoming the limitations of conventional detection algorithms
2Productivity
If traditional image composition evaluation methods are used, then the system can process images, but they fail to capture fine-grained visual details and mutual dependencies between features
Solution Approach 1:
The patent transitions from traditional 2D image processing to 3D feature map representation using FCNs. This dimensional expansion preserves fine-grained visual details and spatial relationships, enabling the system to capture mutual dependencies between features while maintaining processing efficiency through the structured 3D representation
3Extent of automation
If standard convolutional networks are used, then the system can classify images, but they lose fine-grained details during pooling operations
Solution Approach 1:
The patent segments the image processing into distinct functional components: FCN for feature extraction, region composition graph for relationship modeling, and graph convolution for dependency analysis. This segmentation allows each component to preserve and process specific types of information, preventing loss of fine-grained details while maintaining automated classification capability
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosed computer-implemented method may include generating a three-dimensional (3D) feature map for a digital image using a fully convolutional network (FCN). The 3D feature map may be configured to identify features of the digital image and identify an image region for each identified feature. The method may also include generating a region composition graph that includes the identified features and image regions. The region composition graph may be configured to model mutual dependencies between features of the 3D feature map. The method may further include performing a graph convolution on the region composition graph to determine a feature aesthetic value for each node according to the weightings in the node's weighted connecting segments, and calculating a weighted average for each node's feature aesthetic value to provide a combined level of aesthetic appeal for the digital image. Various other methods, systems, and computer-readable media are also disclosed.