Shared-Layer Weight Integration for Distributed Task Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed learning of multilayered neural networks, integrating weights of shared layers across computation nodes can lead to degraded recognition accuracy due to varying progress rates of learning among nodes.
Innovation Solution
An information processing apparatus that updates and evaluates recognition accuracy for each model, sets parameters for integrating shared layer weights based on evaluation results, and integrates these weights using specific parameters to maintain or improve recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If weights are integrated by averaging from multiple recognition models in distributed learning, then the integration process is simple and fast, but recognition accuracy degrades due to differing learning progress rates among computation nodes
Solution Approach 1:
The patent changes the integration parameters by introducing evaluation results as weights in the averaging process. Instead of simple uniform averaging, models are weighted according to their evaluation scores, which reflect their learning progress and accuracy. This parameter change allows the system to maintain fast integration while improving recognition accuracy by giving more influence to higher-performing models.
Solution Approach 2:
The patent implements a feedback mechanism where recognition accuracy is evaluated for each model before integration. These evaluation results are then fed back into the integration process to determine the weighting of each model's weights. This feedback loop ensures that the integration process adapts to the actual performance of individual models, preventing degradation of recognition accuracy while maintaining integration efficiency.
2Ease of manufacture
If weights are integrated uniformly from all recognition models, then the integration process is straightforward, but models with insufficient learning progress negatively impact overall recognition accuracy
Solution Approach 1:
The patent modifies the integration parameters by incorporating evaluation-based weighting. Instead of uniform weighting where all models contribute equally, the integration process uses evaluation results to assign different weights to different models. This parameter change maintains integration simplicity while improving reliability by reducing the negative impact of under-trained models through lower weighting of their contributions.
Solution Approach 2:
The patent applies local quality by treating each model's contribution differently based on its individual performance characteristics. Rather than applying a uniform integration approach to all models, the system evaluates each model's recognition accuracy and assigns localized weighting accordingly. This allows high-performing models to have greater influence while minimizing the impact of lower-performing models on the integrated result.
Data Source
AI summary
There is provided with an information processing apparatus. A first updating unit updates a weight using corresponding learning data, with respect to each of a plurality of recognition models. An evaluating unit evaluates recognition accuracy with respect to each of the recognition models. A first setting unit sets parameters for integrating weights corresponding to a shared layer of the recognition models based on results of evaluation of the recognition accuracy. An integrating unit integrates the weights corresponding to the shared layer based on the parameters.