Precision Propagation in Machine Learning Computation Graphs

The patent applies local quality by assigning different precision levels to different operators within the computation graph based on their specific requirements. Instead of uniformly using high precision across all operators, the system identifies and applies reduced precision (e.g., bfloat16) to operators where it is sufficient, while maintaining higher precision only where necessary for accuracy-critical operations. This localized precision assignment reduces overall memory capacity and bandwidth requirements while preserving computation accuracy where needed.

The patent changes the precision parameter of numerical values from a fixed high-precision setting to variable precision levels (e.g., transitioning from float32 to bfloat16 for certain operators). This parameter change allows the system to reduce memory capacity and bandwidth consumption by using lower precision representations, while the selective application strategy ensures that accuracy is maintained for operations where high precision is critical.

The system applies reduced precision locally to specific operators where it is sufficient, rather than universally. By analyzing the computation graph and identifying operators that can tolerate lower precision (such as certain activation functions or non-critical intermediate computations), the system reduces memory and bandwidth usage in those specific locations while maintaining high precision for accuracy-critical operators, thus resolving the accuracy-loss problem.

The patent applies reduced precision partially to only those operators where it is appropriate, rather than excessively applying it to all operators. This selective partial application ensures that memory capacity and bandwidth are reduced while computation accuracy is preserved for operations that require it, balancing the trade-off between resource efficiency and computational correctness.

The patent performs preliminary precision analysis by propagating precision requirements through the computation graph before execution. By pre-determining which operators can use reduced precision based on downstream operator requirements, the system enables more operators to use efficient reduced-precision formats. This preliminary action, performed during graph compilation or optimization, manages the complexity of precision propagation by resolving it in advance rather than during runtime.

The computation graph itself provides the information needed for precision propagation by analyzing its own structure and operator dependencies. The system uses the graph's inherent data flow information to automatically determine precision requirements, eliminating the need for external complex analysis tools. This self-service approach manages propagation complexity by leveraging the existing graph structure.