Artifact Segmentation for Iterative Inpainting in Modified Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image generation systems lack accuracy and flexibility in generating or modifying digital images, often introducing perceptual artifacts such as broken structures or color blobs, especially in large hole regions or complex structures, and require manual user corrections.
Innovation Solution
An artifact segmentation system using a machine-learning model trained on user-labeled perceptual artifacts to detect and iteratively inpaint synthetic digital images, reducing artifacts by selecting from multiple inpainting models based on artifact ratio metrics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional image generation systems are used to generate or modify digital images, then the basic image editing function is provided, but perceptual artifacts such as broken structures or color blobs are introduced into the synthetically generated portions
Solution Approach 1:
The patent introduces an artifact segmentation model as an intermediary component between the image generation model and the final output. This model detects perceptual artifacts in generated images and generates artifact segmentation maps that guide subsequent inpainting operations, thereby resolving the contradiction between generating images efficiently and avoiding artifacts.
Solution Approach 2:
The system performs preliminary artifact detection and segmentation immediately after image generation, identifying artifact regions before final output. This preliminary action allows the system to prepare targeted inpainting operations, improving generation accuracy without sacrificing efficiency.
2Manufacturing precision
If manual user corrections are required to fix perceptual artifacts, then artifact-free images can be achieved, but the operation time and complexity increase significantly
Solution Approach 1:
The system implements self-service by automatically detecting artifacts, generating segmentation maps, and performing inpainting operations without manual intervention. The artifact segmentation model and iterative inpainting process work autonomously to produce artifact-free images, eliminating the need for manual corrections while maintaining high image quality.
Solution Approach 2:
The system uses feedback from the artifact segmentation model to guide iterative inpainting operations. The segmentation maps provide feedback about artifact locations, which are then used to target specific regions for inpainting, automatically improving image quality without user intervention.
3Productivity
If simple inpainting models are used, then the processing speed is maintained, but the accuracy in complex hole regions or large areas deteriorates
Solution Approach 1:
The patent applies local quality by using artifact segmentation maps to identify specific regions requiring inpainting and selecting or applying appropriate inpainting strategies for each local region. Complex regions with artifacts receive targeted inpainting attention, while simple regions maintain processing efficiency, resolving the contradiction between speed and accuracy.
Solution Approach 2:
The system segments the generated image into artifact-containing regions and artifact-free regions using the artifact segmentation model. This segmentation allows differential processing where computational resources are focused on artifact regions, improving accuracy in complex areas without unnecessarily processing entire images, thus maintaining overall productivity.
4Manufacturing precision
If iterative inpainting processes are implemented to reduce artifacts, then the image quality improves, but the computational resources and processing time increase
Solution Approach 1:
The system performs partial inpainting by focusing computational resources only on artifact-containing regions identified by the segmentation model, rather than re-processing entire images in each iteration. This partial action reduces computational overhead while maintaining the quality improvements of iterative processing.
Solution Approach 2:
By segmenting images into artifact and non-artifact regions, the system enables selective iterative inpainting only where needed. This segmentation approach maintains the beneficial quality improvements of iterative processing while significantly reducing computational resource consumption by excluding artifact-free regions from repeated processing.
Data Source
AI summary
Methods, systems, and non-transitory computer readable storage media are disclosed for generating neural network based perceptual artifact segmentations in synthetic digital image content. The disclosed system utilizing neural networks to detect perceptual artifacts in digital images in connection with generating or modifying digital images. The disclosed system determines a digital image including one or more synthetically modified portions. The disclosed system utilizes an artifact segmentation machine-learning model to detect perceptual artifacts in the synthetically modified portion(s). The artifact segmentation machine-learning model is trained to detect perceptual artifacts based on labeled artifact regions of synthetic training digital images. Additionally, the disclosed system utilizes the artifact segmentation machine-learning model in an iterative inpainting process. The disclosed system utilizes one or more digital image inpainting models to inpaint in a digital image. The disclosed system utilizes the artifact segmentation machine-learning model detect perceptual artifacts in the inpainted portions for additional inpainting iterations.