Mobile Image Object Replacement via Server-Side Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users face difficulties in simulating objects in a physical room using mobile client devices with limited resources, such as small screens and limited processing power, especially when trying to replace existing objects in images or videos.
Innovation Solution
A client-server-based network architecture with a physical item replacement system that includes engines for capture, classification, removal, masking, pose determination, light management, and 3D modeling, allowing users to select and remove objects from images and replace them with virtual items, while accurately positioning virtual light sources to mimic the real-world environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If image processing and object replacement operations are performed on a mobile client device, then users can directly manipulate and replace objects in real-time, but the device's limited processing power and small screen make it difficult to adjust and simulate objects accurately
Solution Approach 1:
The patent introduces a server as an intermediary between the mobile client device and the image processing tasks. The server performs computationally intensive operations such as generating candidate images with different objects, while the mobile device handles user interaction and displays results. This division allows complex processing to occur on powerful server infrastructure rather than constrained mobile hardware.
Solution Approach 2:
The system segments the image processing workflow into distinct components: the mobile client handles capture, display, and user input, while the server handles object removal, candidate image generation, and processing. This segmentation allows each component to operate within its capability constraints while contributing to the overall functionality.
2Adaptability or versatility
If the system generates and displays multiple candidate images with different objects for user selection, then users can choose from more options, but the small screen size makes it difficult to view and compare multiple images effectively
Solution Approach 1:
The system presents candidate images to the user in sequential batches rather than all at once. The user can scroll through or navigate through multiple sets of candidate images, allowing the limited screen space to display a manageable number of options at a time while still providing access to a large total number of candidates through periodic navigation.
Solution Approach 2:
The system transitions from displaying all candidate images simultaneously in a single spatial dimension to organizing them in multiple dimensions (pages, scrolls, or categorized groups). This allows the limited two-dimensional screen space to accommodate a much larger number of candidates by adding navigation dimensions.
3Speed
If the system performs real-time image processing and object replacement on mobile devices, then users can get immediate results, but the limited resources make real-time processing challenging
Solution Approach 1:
The server acts as an intermediary that handles the computationally intensive real-time processing tasks. When the mobile device captures an image or receives user input, it immediately transmits the data to the server, which performs the heavy lifting of object removal, candidate generation, and image processing, then returns results to the mobile device for display.
Data Source
AI summary
A system for replacing physical items in images is discussed. A depicted item can be selected and removed from an image via image mask data and pixel merging techniques. Virtual light source positions can be generated based on real-world light source data from the image. A rendered simulation of a virtual item can then be integrated into the image to create a modified image for display.