Asynchronous Multi-User Image Editing via GAN Latent Space

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional image editing systems using neural networks, such as GANs, are computationally intensive and require significant resources, limiting real-time image editing capabilities, especially on less powerful devices like mobile devices, due to inefficient resource utilization and slow processing speeds.

Innovation Solution

An improved image editing system employs a web-based intermediary with a multi-process container supporting multiple users, utilizing dual layered memory for concurrent projection and transformation operations, clubbing transformation requests, and implementing arithmetic increment for smooth display updates, allowing multiple users to perform asynchronous image editing operations in real-time.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If conventional image editing systems use GANs to modify images, then image editing quality is improved, but computing resource consumption increases significantly

Engineering Contradiction:

Improveimage editing qualityVSAvoidcomputing resource consumption

Core Design Contradiction:

Manufacturing precisionVSUse of energy by moving object

Solution Approach 1:

The system segments the image editing process into two distinct neural network components: an encoder network that extracts features from the input image, and a generator network that applies edits based on those features. This segmentation allows each network to be optimized for its specific function, improving overall efficiency while maintaining quality.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system creates a latent representation (feature vector) that is a compressed copy of the essential image characteristics. Instead of processing the entire high-resolution image through both networks, the generator works with this compact latent representation, significantly reducing computational requirements while preserving editing quality.

Inventive Principle:

Principle #26Copying

2Manufacturing precision

If conventional systems process images using GANs, then image transformation accuracy is improved, but processing speed deteriorates

Engineering Contradiction:

Improveimage transformation accuracyVSAvoidprocessing speed

Core Design Contradiction:

Manufacturing precisionVSSpeed

Solution Approach 1:

The encoder network performs preliminary processing by extracting and compressing image features into a latent representation before the generator network applies edits. This preliminary action reduces the dimensionality and complexity of the data that the generator must process, enabling faster transformation while maintaining accuracy.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system extracts only the essential features of the image that are relevant to the editing task, discarding redundant information. This extraction is performed by the encoder network which produces a compact latent vector containing only the necessary information for accurate image transformation, thereby speeding up processing.

Inventive Principle:

Principle #2Taking out (Extraction)

3Speed

If image editing operations are performed on devices with powerful processors, then real-time editing capability is improved, but device compatibility worsens

Engineering Contradiction:

Improvereal-time editing capabilityVSAvoiddevice compatibility

Core Design Contradiction:

SpeedVSAdaptability or versatility

Solution Approach 1:

The system changes the parameter of image representation from high-resolution pixel data to a compressed latent representation. This parameter change reduces the computational burden significantly, allowing the same system to achieve real-time editing performance across a wider range of devices with varying processing capabilities.

Inventive Principle:

Principle #35Parameter changes

4Manufacturing precision

If the encoder network progressively projects the image into latent space with multiple iterations, then latent vector optimization is improved, but processing time increases

Engineering Contradiction:

Improvelatent vector optimizationVSAvoidprocessing time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The system performs a limited number of progressive projection iterations rather than exhaustively optimizing the latent vector. This partial action provides sufficient optimization for practical editing purposes while avoiding the excessive processing time that would result from continued iterations, achieving a practical optimum balance.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11928762B2Asynchronous multi-user real-time streaming of web-based image edits using generative adversarial network(s)

Publication Date: 2024.03.12 ADOBE INC

AI summary

Embodiments of the present invention provide systems, methods, and computer storage media for editing images using a web-based intermediary between a user interface on a client device and an image editing neural network(s) (e.g., a generative adversarial network) on a server(s). The present image editing system supports multiple users in the same software container, advanced concurrency of projection and transformation of the same image, clubbing transformation requests from several users hosted in the same software container, and smooth display updates during a progressive projection.