Learning Program for Deep Neural Network Decimal Point Adjustment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In deep learning, dynamic changes in the computation graph during machine learning lead to incorrect detection of calculation routes, resulting in decreased learning accuracy due to the reference of wrong statistical information for adjusting the decimal point position in 8-bit fixed-point calculations.
Innovation Solution
A learning program and method that dynamically generates a key based on the calculation route followed by tensors, allowing the reference of appropriate statistical information from past iterations, thereby adjusting the decimal point position accurately and maintaining learning accuracy even with changes in the computation graph or node structures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If 8-bit fixed-point calculations are used to improve computation speed and power efficiency, then calculation speed and power performance are improved, but learning accuracy decreases due to incorrect decimal point position adjustment
Solution Approach 1:
The system performs preliminary actions by storing route information and statistical information from past iterations before the current calculation. When executing calculations in the current iteration, the system retrieves pre-stored statistical information corresponding to the current calculation route, enabling accurate decimal point position adjustment without performing extensive real-time analysis, thus maintaining both speed and accuracy.
2Adaptability or versatility
If the computation graph structure changes dynamically during learning, then adaptability to different learning stages is improved, but incorrect statistical information is referenced leading to decreased learning accuracy
Solution Approach 1:
The system dynamically adapts to changing computation graph structures by generating route information that reflects the current graph configuration. Statistical information is organized and retrieved based on this dynamic route information, ensuring that the system automatically adjusts to match the current computation graph structure while maintaining accurate decimal point position adjustment, thus resolving the contradiction between adaptability and accuracy.
3Reliability
If statistical information from past iterations is stored and referenced, then learning accuracy is improved through accurate decimal point adjustment, but memory resources and storage complexity increase
Solution Approach 1:
The system applies local quality by storing and retrieving only the specific statistical information relevant to each calculation route. Instead of storing all possible statistical information universally, the system organizes statistical data locally according to route information, retrieving only what is needed for the current calculation path. This selective approach reduces memory consumption while maintaining learning accuracy.
Data Source
AI summary
A non-transitory computer-readable recording medium stores therein a learning program for causing a computer to execute a process comprising: referring to, at time of learning a computation model that is a target of deep learning and has a plurality of nodes, a storage unit in which route information that indicates a calculation route followed by a tensor in each stage of learning prior to the time of learning, and statistical information regarding a position of a decimal point used in the calculation route are associated with each other; acquiring, when executing each piece of calculation processing set in each of the plurality of nodes at the time of learning, the statistical information corresponding to the route information that reaches each of the plurality of nodes; and executing the each piece of calculation processing using the position of the decimal point specified by the acquired statistical information.