Symbolic Differentiation via Derivative Graph Factorization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic differentiation techniques, such as finite differencing, automatic differentiation, and symbolic differentiation, are inefficient and complex, especially for functions with multiple inputs and outputs, leading to increased computation time and memory exhaustion or inordinate processing times.
Innovation Solution
The efficient symbolic differentiation method represents functions as expression graphs, generates derivative graphs, and factors them to reduce redundancy, allowing for the computation of derivatives by summing products along all product paths in the factored graph, which is asymptotically more efficient as function complexity increases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional symbolic differentiation is used, then derivatives can be computed for simple functions, but computation time and memory requirements increase exponentially as function size and complexity increase
Solution Approach 1:
The patent segments the differentiation process by introducing intermediate derivative nodes that break down the complex differentiation task into smaller, manageable sub-tasks. Each intermediate node represents a partial derivative computation, allowing the system to process complex functions through a hierarchical structure rather than attempting to compute all derivatives simultaneously, thus reducing overall computation time and memory requirements.
Solution Approach 2:
The patent extracts common sub-expressions and shared derivative computations from the differentiation graph. By identifying and extracting common factors that appear multiple times in the differentiation process, the system avoids redundant computations and reduces both time and memory complexity. This extraction is achieved through graph algorithms that detect and eliminate duplicate sub-trees in the expression graph.
2Adaptability or versatility
If automatic differentiation is used for functions with multiple inputs and outputs, then derivatives can be computed, but the process becomes inefficient and requires specifying forward or reverse mode which increases complexity
Solution Approach 1:
The patent implements a universal differentiation approach that handles all function types (single input/output, multiple inputs/single output, single input/multiple outputs, and multiple inputs/outputs) through a single unified algorithm. The system automatically detects function signatures and applies the appropriate differentiation strategy without requiring users to manually specify forward or reverse mode, reducing operational complexity while maintaining versatility.
Solution Approach 2:
The patent dynamically adapts the differentiation strategy based on the detected function signature. The system automatically determines whether to use forward mode, reverse mode, or a combination thereof, and dynamically adjusts the computation graph construction accordingly. This dynamic adaptation eliminates the need for manual mode specification while optimizing performance for each function type.
3Productivity
If automatic differentiation is used, then derivatives can be computed simultaneously with function evaluation, but memory exhaustion occurs for complex functions
Solution Approach 1:
The patent segments the computation graph into manageable portions by introducing intermediate derivative nodes that divide the large computation into smaller sub-computations. This segmentation allows the system to process and store only necessary portions of the computation graph in memory at any given time, reducing peak memory consumption while maintaining high computation speed through efficient parallel processing of independent sub-tasks.
Solution Approach 2:
The patent extracts and eliminates redundant computational paths and shared sub-expressions from the computation graph. By identifying and removing duplicate computations that consume excessive memory, the system reduces the overall memory footprint while preserving the essential derivative computation pathways, enabling handling of complex functions without memory exhaustion.
Data Source
AI summary
An efficient symbolic differentiation method and system that automatically computes one or more derivatives of a function using a computing device. A derivative graph is used to graphically represent the derivative of a function. Repeated factorization of the derivative graph yields a factored derivative graph. The derivative is computed by summing the products along all product paths in the factored derivative graph. The efficient symbolic differentiation method and system operates on both single input/single output and multiple input/multiple output functions. For a single input/single output function, the order of the factoring does not matter. However, for a multiple input/multiple output function, the factoring order is such that the factor subgraph appearing most frequently in the derivative graph is factored first. The method and system also use a product pairs priority queue to avoid the re-computing of sub-strings that are common between product paths.