Multi-Layer Neural Network Shared Layer Determination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep convolutional neural networks (DCNNs) face challenges in finding the optimal network structure, require large memory capacity for learning parameters, and take a long time for recognition processing due to the need for simultaneous execution of multiple recognition tasks, which complicates the sharing of learning parameters between tasks.
Innovation Solution
An information processing apparatus that includes a learning unit for multi-layer neural networks, a generation unit for shared layers, a relearning unit to refine the network structure using shared layers, and a determination unit to evaluate the sharing of these layers across tasks, reducing the number of learning parameters and processing time by identifying suitable shared layers for each task.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple recognition tasks are learned simultaneously using a single multi-layer neural network, then the recognition performance is improved by mutually using common knowledge, but the memory capacity requirement increases due to the large number of learning parameters
Solution Approach 1:
The patent segments the multi-layer neural network into multiple independent single-task neural networks, each dedicated to a specific recognition task. This segmentation allows each network to have a optimized, smaller parameter set while maintaining high recognition performance for its specific task, thereby reducing the total memory capacity requirement compared to a single large multi-task network.
Solution Approach 2:
The patent creates a system where multiple single-task neural networks work together to achieve multi-task recognition capability. Each network is specialized for one task but the collective system provides universal multi-task functionality, allowing the system to maintain high performance across multiple tasks without requiring each network to be universally designed for all tasks simultaneously.
2Measurement precision
If multiple recognition tasks are learned simultaneously using a single multi-layer neural network, then the recognition performance is improved by mutually using common knowledge, but the processing time increases due to the complexity of learning and evaluation
Solution Approach 1:
By dividing the learning process into separate single-task neural networks, each network can be trained independently and more efficiently on its specific task. This segmentation avoids the computational overhead of simultaneously updating parameters for multiple tasks in a single network, reducing the overall learning time while maintaining or improving recognition performance through task-specific optimization.
3Measurement precision
If the network structure is thoroughly searched to find the optimum configuration for multi-task learning, then the recognition performance is optimized, but the learning time becomes prohibitively long
Solution Approach 1:
The patent avoids exhaustive search of the entire multi-task network structure by segmenting into multiple single-task networks. Each single-task network has a simpler, more focused structure that requires less computational search effort. The overall optimal configuration is achieved by combining these individually optimized networks, thereby obtaining high recognition performance without prohibitively long learning times.
Data Source
AI summary
An information processing apparatus includes a learning unit configured to learn a plurality of multi-layer neural networks configured to carry out a plurality of tasks, a generation unit configured to generate a shared layer candidate at a predetermined layer between or among the plurality of multi-layer neural networks, a first relearning unit configured to relearn the plurality of multi-layer neural networks in a structure using the shared layer candidate, and a determination unit configured to determine whether to share the shared layer candidate at the predetermined layer with respect to each of the plurality of tasks based on an evaluation of the relearning.