Mobile Image Cleanup via Multiscale Foreground Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image cleanup methods require significant computer processor and memory resources, making them inefficient for use on resource-constrained mobile devices, which are needed to distribute the burden from backend server systems and improve image processing efficiency.
Innovation Solution
Implementing a method on user equipment that subsamples images using a multiscale transform, identifies foreground and background portions via machine learning models, generates and upscales masks to remove background portions, and applies these masks to the original images, thereby reducing resource requirements and enhancing image cleanup efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional image cleanup methods are used on mobile devices, then image processing quality can be maintained, but computer processor and memory resources are excessively consumed
Solution Approach 1:
The image processing task is segmented into two stages: (1) subsampling the image to a lower resolution to perform initial foreground-background segmentation, and (2) upsampling the generated mask back to the original resolution. This segmentation allows the computationally intensive classification task to be performed on a smaller image, reducing processor and memory resource consumption while maintaining acceptable processing quality.
Solution Approach 2:
Instead of processing the full-resolution image directly, the patent creates a subsampled copy of the image at a lower resolution for the machine learning classification step. The resulting mask is then copied back to the original image resolution through upsampling. This copying approach enables resource-constrained mobile devices to perform image cleanup effectively.
2Power
If image cleanup is performed on backend server systems, then processing power is sufficient, but the burden on backend systems increases and efficiency decreases
Solution Approach 1:
The patent extracts the image cleanup functionality from the backend server system and implements it directly on the mobile device (user equipment). By taking out this processing task from the backend, the system reduces the burden on server infrastructure and improves overall processing efficiency, as the mobile device can perform cleanup operations locally without requiring extensive backend processing.
3Measurement precision
If full-resolution images are processed directly, then processing accuracy is maximized, but processing time and resource consumption increase significantly
Solution Approach 1:
The patent changes the resolution parameter of the image during processing. The image is subsampled to a lower resolution before machine learning classification, which reduces processing time and resource consumption. After the mask is generated, it is upsampled back to the original resolution. This parameter change strategy balances classification accuracy with processing efficiency.
Data Source
AI summary
Methods, systems, and articles of manufacture, including computer program products, are provided for image cleanup. In some embodiments, there is provide a method which may include subsampling a first image to a first level image of a multiscale transform; performing, based on a machine learning model, an identification of a foreground portion of the first level image and a background portion of the first level image; generating, based on the identification of the foreground portion and the background portion, a first mask; upscaling the first mask to a resolution corresponding to the first image depicting the foreground item; applying the upscaled first mask to the first image to form a second image depicting the foreground item; and providing the second image depicting the foreground item to a publication system. Related systems and articles of manufacture, including computer program products, are also provided.