Tree-Like Deep Neural Network Training for Multi-Task Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning methods are inefficient for multiple tasks due to high redundancy in neural networks, requiring separate calculations and combining results, which is time-consuming.
Innovation Solution
A deep neural network training method that utilizes a tree-like network topology with leaf nodes trained for specific tasks, clustering nodes by task attributes to form parent nodes sharing common attributes, reducing redundancy through a signal control mechanism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple neural networks are used to perform calculations for multiple tasks separately, then each task can be handled independently, but the operation efficiency decreases due to high redundancy and time-consuming processes
Solution Approach 1:
The patent merges multiple separate neural networks into a single unified neural network that can handle multiple tasks simultaneously. The network shares common layers and parameters across different tasks, eliminating redundancy while maintaining the ability to perform multiple functions. This is achieved through a multi-task learning framework where task-specific layers are added to a shared base network.
Solution Approach 2:
The patent creates a universal neural network architecture that serves multiple purposes. The shared components of the network are designed to be task-agnostic and can be reused across different tasks, making the system multi-functional. This universal structure allows the same network to perform various tasks by activating different task-specific pathways while sharing computational resources.
2Reliability
If multiple separate neural networks are used for multiple tasks, then each task receives dedicated processing, but the system complexity increases due to redundancy
Solution Approach 1:
The patent combines multiple task-specific neural networks into one integrated network, reducing overall system complexity. By merging shared processing components and eliminating duplicate structures, the patent maintains reliable task processing while simplifying the architectural complexity. The unified network structure reduces the number of independent components that need to be managed and trained.
3Manufacturing precision
If separate neural networks are used for each task, then task-specific optimization is achieved, but training time increases due to redundant computations
Solution Approach 1:
The patent merges the training processes of multiple tasks into a unified training framework. By combining gradient computations and parameter updates across tasks, the patent eliminates redundant training computations. The shared network components are trained once and benefit all tasks, significantly reducing total training time while maintaining task-specific performance through task-specific loss functions and layers.
Data Source
Figure 1
Figure 2~3
AI summary
A deep neural network training method and apparatus and a computer device are provided. The deep neural network training method includes: obtaining task attributes of nodes in a current network layer in a tree-like network topology (S101); performing cluster analysis on the nodes in the current network layer based on the task attributes of the nodes in the current network layer and extracting a common part of task attributes of multiple nodes in a same category as a task attribute for a parent node of the multiple nodes (S102); training a network parameter of each parent node based on a task attribute of this parent node (S 103); and determining that training of a deep neural network corresponding to the tree-like network topology is completed, after completion of training of all nodes in all network layers (S 104). The operation efficiency of deep learning can be improved through this solution.