Multi-task Stacked Neural Network for Training Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multi-task machine learning networks face challenges in task balancing and optimizing training for multiple tasks simultaneously, particularly in image analytics, where each task requires appropriate weighting and prior information sharing between tasks.
Innovation Solution
A multi-task stacked neural network architecture is introduced, comprising multiple sub-networks that sequentially solve individual tasks by leveraging prior information from previous tasks, with a general input connecting input nodes of sub-networks and output nodes of one sub-network serving as input for subsequent ones, allowing for end-to-end training and mitigating task balancing issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple tasks are solved simultaneously in a single multi-task network, then task balancing becomes challenging and training optimization becomes difficult, but solving tasks independently requires multiple trainings and increases redundancy
Solution Approach 1:
The patent segments the multi-task learning problem into multiple single-task sub-networks, where each sub-network is responsible for solving one specific task. This segmentation allows each sub-network to be trained independently on its own task without the complexity of balancing multiple tasks simultaneously, while still achieving multi-task functionality through the combination of sub-networks.
Solution Approach 2:
The patent creates a universal framework where multiple sub-networks work together to solve multiple tasks. Each sub-network is specialized for a specific task, but the overall system achieves multi-functionality by combining the outputs of all sub-networks, thus avoiding the need for complex task balancing while maintaining productivity.
2Reliability
If a dedicated segmentation algorithm is trained for every landmark independently, then each task can be solved with sufficient quality, but multiple trainings are required and parameter space redundancy increases
Solution Approach 1:
The patent segments the learning process into independent sub-networks, each dedicated to a specific task. This allows each sub-network to achieve high solution quality for its specific task while training can be performed in parallel, significantly reducing the total training time compared to sequential independent training.
Solution Approach 2:
The patent enables continuous and parallel training of multiple sub-networks simultaneously. Instead of training each sub-network sequentially (which would waste time), all sub-networks can be trained in parallel using different batches of data, maintaining continuous useful action and reducing overall training time while preserving task solution quality.
3Adaptability or versatility
If instance segmentation is used to jointly determine bounding boxes and classes, then object detection and segmentation can be combined, but many unsolved technical challenges remain
Solution Approach 1:
The patent segments the complex instance segmentation problem into simpler, dedicated sub-tasks, each handled by a specialized sub-network. This segmentation avoids the technical challenges of implementing full instance segmentation while still achieving the benefit of combining multiple tasks through the modular sub-network architecture.
Solution Approach 2:
The patent extracts and separates the complex instance segmentation task into individual simpler tasks, each solved by a dedicated sub-network. This extraction removes the technical challenges associated with implementing complex instance segmentation while retaining the versatility of handling multiple tasks through the modular architecture.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
The invention describes a multi-task machine learning network (1) comprising a plurality of sub-networks (N1, N2, N3, N4), each sub-network (N1, N2, N3, N4) having a number of output nodes (ON) and input nodes (IN), wherein there is a general input (GI) providing input data (D) and the input nodes (IN) of at least a first sub-network (N1) and a second sub-network (N2) are connected to this general input (GI), and wherein an output-node (ON) of the first sub-network (N1) is connected with an input-node (IN) of the second sub-network (N2). The invention further describes a training method for such a network as well as a medical system.