Optical Flow Computation Circuitry for Dense Video Analysis

The patent segments the optical flow computation into distinct functional units: gradient computation units, matrix construction units, and incremental update units. Each unit processes specific portions of the computation independently, allowing parallel operation and reducing the need for centralized external memory storage. The image data is divided into windows processed separately, with intermediate results held in on-chip registers.

The patent transitions from a memory-bandwidth-limited architecture to a compute-bound architecture by adding temporal dimension through incremental updates. Instead of recomputing gradient matrices from scratch for each pixel, the system maintains state in on-chip registers and performs incremental updates, effectively adding a time dimension to the computation process that eliminates repeated memory accesses.

The patent performs preliminary computation of gradient values and stores them in on-chip registers before they are needed for matrix construction. The gradient computation units pre-calculateIx and Iy values and hold them in registers, so that when matrix construction is needed, the data is already available on-chip without requiring external memory access or additional arithmetic units for repeated computation.

The patent discards the conventional approach of storing complete gradient matrices in external memory and instead recovers efficiency by maintaining only essential intermediate results in on-chip registers. The system discards redundant computations by using incremental updates that recover and reuse previous computation results, keeping only the necessary state information in fast on-chip storage.

The patent merges multiple computation steps into unified on-chip operations. The gradient computation, matrix construction, and optical flow calculation are merged into a single pipeline that operates entirely on on-chip data. This consolidation eliminates the need for intermediate storage in external memory and reduces memory traffic to only the initial image data input and final results output.

The system makes itself self-sufficient by maintaining all necessary intermediate computation results in on-chip registers. The incremental update mechanism allows each computation stage to serve itself with data from previous stages without external memory intervention. The system serves its own data needs through internal register transfers and incremental updates, eliminating dependence on external memory bandwidth.