Multi-task Fusion Neural Network Architecture for Mobile Image Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks for image processing tasks on mobile devices require multiple networks with separate feature extractors, leading to increased complexity and inefficiency, as they are typically trained for specific tasks and lack the ability to perform tasks concurrently or leverage task inter-relationships.
Innovation Solution
A multi-task fusion neural network architecture that uses a common front-end feature extractor with shared weights for multiple tasks and a back-end fusion layer to combine outputs from different task-specific branches, allowing concurrent processing and information fusion across tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple separate convolutional neural networks are used for different image processing tasks, then each task can be performed with task-specific optimization, but the overall system complexity increases and processing efficiency decreases
Solution Approach 1:
The patent merges multiple separate convolutional neural networks into a single multi-task fusion neural network. The network shares a common feature extractor across all tasks and uses a back-end fusion layer to combine task-specific processing, thereby reducing system complexity while maintaining task-specific optimization capabilities.
Solution Approach 2:
The patent creates a universal neural network architecture that can perform multiple image processing tasks simultaneously. The common feature extractor serves all tasks universally, while task-specific branches provide specialized processing, enabling the system to handle depth estimation, segmentation, and other tasks with a single multi-functional network.
2Measurement precision
If multiple separate convolutional neural networks are used for different image processing tasks, then each network can be trained independently for its specific task, but the processing speed and concurrency are reduced
Solution Approach 1:
The patent combines multiple independently trained networks into a single unified network that processes multiple tasks concurrently. The shared feature extractor enables simultaneous task processing while the back-end fusion layer maintains task-specific processing capabilities, improving processing speed without sacrificing training accuracy.
Solution Approach 2:
The patent enables continuous concurrent processing of multiple tasks through the unified network architecture. Instead of sequentially processing tasks through separate networks, the multi-task fusion network processes all tasks simultaneously in a continuous manner, improving overall productivity while maintaining task-specific accuracy through the fusion layer.
3Measurement precision
If task-specific neural networks are used, then each task can be optimized independently, but the ability to leverage task inter-relationships is lost
Solution Approach 1:
The patent merges task-specific networks while preserving task inter-relationships through the back-end fusion layer. This layer combines the outputs of task-specific branches, enabling the network to leverage relationships between tasks (such as using depth information to improve segmentation) while maintaining independent optimization capabilities for each task.
Data Source
AI summary
A method includes identifying, by at least one processor, multiple features of input data using a common feature extractor. The method also includes processing, by the at least one processor, at least some identified features using each of multiple pre-processing branches. Each pre-processing branch includes a first set of neural network layers and generates initial outputs associated with a different one of multiple data processing tasks. The method further includes combining, by the at least one processor, at least two initial outputs from at least two pre-processing branches to produce combined initial outputs. In addition, the method includes processing, by the at least one processor, at least some initial outputs or at least some combined initial outputs using each of multiple post-processing branches. Each post-processing branch includes a second set of neural network layers and generates final outputs associated with a different one of the multiple data processing tasks.