Local Convolutional Neural Network for Mobile Image Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile client devices with limited computing resources struggle to perform complex image processing tasks efficiently, leading to long processing times and increased power consumption, which negatively impacts user experience and battery life.
Innovation Solution
Implementing a local convolutional neural network on mobile devices to perform real-time or near-real-time image and video stream modifications, such as face manipulation, by using efficient convolutional kernel approximations and tensor projections, allowing for fast inference speed, compact model size, and low energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If complex image processing is performed on mobile client devices, then image modification capability is improved, but power consumption increases and processing time extends
Solution Approach 1:
The patent replaces traditional mechanical image processing methods with neural network-based processing. The neural network model, trained on server-side computational resources, is deployed to mobile devices to perform complex image modifications such as face manipulation, style transfer, and object detection. This substitution enables mobile devices to execute sophisticated image processing tasks that would otherwise be computationally infeasible on such hardware, directly improving image modification capability while managing power consumption through efficient model design.
Solution Approach 2:
The patent employs parameter optimization techniques to adjust the neural network model for mobile deployment. By modifying model parameters such as reducing layer depth, decreasing filter counts, and optimizing activation functions, the system achieves a balance between processing capability and power consumption. These parameter changes enable the neural network to perform complex image modifications on mobile devices with limited computational resources and battery capacity.
2Adaptability or versatility
If complex image processing is performed on mobile client devices, then image modification capability is improved, but processing time increases
Solution Approach 1:
The patent applies preliminary action by pre-training the neural network model on extensive image datasets using powerful server-side computational resources before deploying it to mobile devices. The model is pre-processed and optimized during the training phase, allowing it to perform complex image modifications efficiently on mobile hardware without requiring real-time training. This preliminary preparation enables fast inference speed on resource-constrained devices.
Solution Approach 2:
The patent replaces traditional mechanical image processing algorithms with a neural network-based system that has been optimized for mobile deployment. By substituting conventional processing methods with a pre-trained neural network, the system achieves faster processing times for complex tasks such as face manipulation and style transfer, as the neural network can leverage learned patterns rather than computing from scratch.
3Productivity
If image processing is passed to networked computing devices, then processing capability is improved, but device complexity and network dependency increase
Solution Approach 1:
The patent extracts the essential image processing functionality from complex server-side systems and encapsulates it in a standalone neural network model that can run independently on mobile devices. By taking out the core processing capability and packaging it as a portable model, the system reduces network dependency and simplifies the overall architecture while maintaining high processing capability. The mobile device becomes self-sufficient for image modifications without requiring constant network communication.
Solution Approach 2:
The patent creates a universal neural network model that can perform multiple image processing functions on mobile devices, including face detection, manipulation, style transfer, and object detection. This multi-functional model replaces the need for multiple specialized processing systems, reducing device complexity and eliminating network dependency while maintaining comprehensive processing capability across various image modification tasks.
Data Source
AI summary
A system of machine learning schemes can be configured to efficiently perform image processing tasks on a user device, such as a mobile phone. The system can selectively detect and transform individual regions within each frame of a live streaming video. The system can selectively partition and toggle image effects within the live streaming video.


